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ABSTRACT 

BA Previews, the naohine-xeadable vexslon of Biological Abetraota and 
BioReaeaaah Indajc. has been uaed to provide an experinental S.B.I, service. 
Boring a two-year period (1970, 1971) S.B.I, search profiles were written 
for 353 biologists and other research workers with a need for biological 
information in academic, industrial and government research institutions. 
Profiles were also stqpplied to ei^t information centres. *fhree •liaison 
scientists* were responsible for recruiting the eacperimental population, 
and for interviewing participants in order to identify their information 
requirements, to construct and amend their search profiles, and to discuss 
their reactions to the serviced At the beginning of^the experiment a 
questionnaire/interview survay was made of the information resources then 
being used by participants, with particular reference to their usage of the 
printed Biological Abstracts . 

^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^mm 

'Special studies of the different searchable indexes (enriched titles, 

CROSS codes, BioSystematic codes) on the BA Previews tapes were made by the 

liaison scientists in order to increase their knowledge of the data base and 

hence improve the methods of profile construction.^ 

* « 

A quantitative description of service performance was provided by the 
measurement of the following parameters t output sizes, recall ratios, precision 
ratios, coverage ratios, and novelty ratios. Ihe relevance predictability of ^t,^ 
the enriched titles which appear on the ^ Previews tapes was also studisd. The 
users* reactions to the service were assessed by means of a qiiestionnaire/ 
interview survey at the end of the experiment. Correlations between 
quantitative performance measures and the users assessment of the service, as 
revealed in questionnaire replies, were examined. 

It was concluded that the results obtained in the project Justified a 
further experiment in which biologists would be asked to make a parent toward 
the cost of the service, in addition to providing feerlback. 
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CHAPTER 1 
IlfrRODUCTIQN 

1. ORIGINS OF THE EXPERIMENT 

BA Previews is the machine-readable version of the printed publications. 
Biological Abstracts and BioResearch Index. Both the printed and the 
magnetic tape versions are published by the Biosciences Information Service 
(BIOSIS), Philadelphia, U.S.A. 

Three £A Previews tapes are issued each month: two correspond to 
the semi-monthly issues of Biological Abstracts and one to the monthly 
issue of BioResearch Index. With the exception of the abstracts, all the 
information included in the printed publications is present on the tapes 
and is available for computer searching. 

This report describes the development and evaluation of an experimental 
S.D.I, service based on the BA Previews data base. The project was 
financed by grants from the Office for Scientific and Technical Information 
(OSTi). Prior to the award of these grants, interest in an investigation 
of the BA Previews service had been expressed by members of the Committee 
on Biological Information (COBI) under its" then Chairman, Dr G.C. Ainsworth. 
COBI, through the General Secretary of the Institute of Biology, Dr D.J.B. Copp, 
sought opinion and collaboration from a wide spectrum of biologists. As 
a result of the encouraging response, it was decided to proceed with the 
experimental study described in this report. The Committee has since 
maintained its interest on behalf of the Institute of Biology, the 
Biological Council and Aslib, and nominated three of its members to serve 
on the project's advisory committee. 

2. EXPERIMENTAL OBJECTIVES 

The experiment had the following objectives t 

1. To introduce the biological community to a mechanised information 
service and to test its reaction to this type of information source. 

2, To investigate methods of formulating user profiles for searching 
the BA Previews tapes, making full use of all the searchable indexes 
offered by the data base. 

Selective dissemination of Information, 
o - 
ERIC 
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3. To measure the performance of the retrieval system in terms of 
recall, precision and novelty ratios, and similar parameters. 

4. To draw conclusions as to the potential demand amongst biologists 
for a mechanised S.D.I, service and the suitability of BA Previews 
as the basis for such a service. 

In addition, there were two secondary objectives : 

5. To collect data on the patterns of literature usage of the 
biologists participating in the experiment. 

6. To identify the level of interest in any possible future service 
offering retrospective searches of the BA Previews tapes. 

Although the project was direyted towards the achievement of specific 
experimental objectives, the service was operational in the sense that it 
attempted to satisfy the real information needs of research biologists 
and thereby make a genuine contribution to their research activity. 

The experimental service had two features, which were essential to the 
experiment, but are unlikely to be present in a permanent service : 

a) The use of liaison scientists to provide the interface 
between the user and the service. • 

b) The provision of a free service. 

3. QITTLINE OF THE EXPERIMEIfr 

The staff appointed for the project were three liaison scientists, 
two located in the Experimental Information Unit at Oxford University and 
one in the Department of Botany at Nottingham University. They were 
responsible for the recruitment of participants, the construction and amend- 
ment of user profiles and the collection of feedback from users. Each 
liaison scientist had a first degree in a biological subject; two (b. Smith, 
J. Stow) had postgraduate research experience and one (M. Williams) had 
extensive experience in librarianship and information work. 

Prior to the recruitment of the main population of users, 55 
biologists in the host universities of Oxford and Nottingham took part in 
a pilot scale study during January-March 1970. The purpose of this study 
was to provide the liaison scientists with practical experience in 

ERIC 
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techniques of interviewing and profile construction. 

With the exception of the host universities, the invitation to 
participace in the experiment was addressed to the librarians or information 
officers in academic, industrial and government research institutions. 
Data collected in a government survey of the employment of biologists was 
used as a guide to the selection of a population which was reasonably 
representative with respect to both the employment categories and the 
subject interests of biologists. Preference was given to establishments 
within easy travelling distance from Oxford or Nottingham, but users in 
more distant locations were included if they had expressed a very strong 
interest in the experiment or if they belonged to under-represented subject 
fields. Eight information centres took part in the experiment, thereby 
enabling an assessment to be made of the potential use of the BA Previews 
service as an information source for retrieving references which could be 
included in their own abstracting or indexing publications (See Chapter VI )• 

The personal interview of the liaison scientist with the user was a 
characteristic of the experiment. With a few exceptions, three interviews 
were arranged: the initial interview was for the purposes of identifying 
the user's information needs and usages and constructing a user profile; 
the second interview provided an opportunity to optimise the initial 
profile; and, in the final interview, the user's reactions to the service 
were assessed by means of a questionnaire. 

Recruitment of the user population (353 users; 392 profiles) was 
completed in December 1970. With the exception of 59 users who withdrew 
from tue experiment, all participants continued to receive three computer 
printouts per month until May 1972. These printouts corresponded to 
the three BA Previews tapes received each month from BIOSIS. As a sequel 
to the present experiment, an OSTI-supported study, incorporating charges 
to the user, was established at the Department of Botany, University of 
Nottingham. All participants were given the opportunity to continue their 
profiles for a further ten months at a cost of £10 and 151* elected to do so* 
In the second year of this experiment the price will be increased to £25 
per annum* 
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In order to provide a better understanding of the data base, studies 
were undertaken by the liaison scientists concerning the allocation of 
CROSS and BioSystematic codes, and of "eririchraent terms". These studies 
made a significant contribution to the optimisation of user profiles. In 
July 1970, before the main recruitment took place, M. Williams spent a 
week at BIOSIS in Philadelphia in order to acquire detailed information 
concerning editorial procedures . 

At the first interviews with the users, a questionnaire survey was 
carried out by the liaison scientists to collect data on the users* 
existing literature searching habits and, in particular, their attitude 
to Biological Abstracts. A final questionnaire survey was conducted during 
the latter stages of the experiment (October-November 1971) to obtain 
information on the users* reactions to the BA Previews service, its 
impact on their previous methods of current awareness searching, and their 
likely interest in any future permanent service offering current awareness 
and retrospective search facilities. 

Throughout the experiment, all participants were asked to return 
feedback cards, stating the number of relevant references (MAJOR VALUE and 
MINOR VALUE) retrieved by their profile. The^e cards also provided an 
opportunity for users to comment on the service or to suggest amendments 
to their profile. 

However, for the measurement of service performance, much more 
detailed feedback was required and not all users could be expected to spare 
the time required for the provision of this additional information. 
Approximately half the population (l51 users) volunteered to take part in 
this quantitative evaluatior. phase of the project, which commenced in 
April 1971. The collection of evaluation data was completed in January 
1972. While the evaluation was in progress no 'i-nendments to profiles were 
•Allowed. 

Output size, precision and recall were the parameters used to define 
retrieval perfor.-nance. Coverage, novelty ratios .-ind relevmce predict- 
•ihiiity were -ilso measured. Eptixcttes for the performance ratios were 
b-a:-ed on user judgements of the relevan.e (HA.70R VALUE, MINOR VALUE, 
IRHKLEVAIH') of Individual articles. 



Report of vizil to the United states from . July to 6 Aup;ust. I y/0 
I. Report submitted to OoTlJ, by Margaret I. Williams. Oxford. 1970. 1-, 
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The above parajneters define the performance, not of tho BA Previevs 
data base, but of *an operational S.D#I, service, based on tho BA Previevs 
data base** The values obtaixxed for recall and precision ratios are 
therefore dependent on several variables corresponding to the main 
components which comprise the complete retrieval service* Thus, retrieval 
performance of a particular profile will be detemined by the effective- 
ness of communication between the uaer and the liaison sciontist, the 
skill exercised in profile construction, the flexibility of the search 
logic, and the quality of the indexes provided by the data base* The use 
of "failure analysis" techniques ^^""^^^ would be necessaiy to determine the 
relative impact on retrieval performance of each of these factors. 
However, the resoiirces devoted to the project did not allow a detailed 
analysis of this type to be undertaken. 

h. ADVISORY COMMITTEE Fl - BA PREVIEWS PROJECT 

An Advisory Committee was set up to advise on the conduct of the 
project and to assist in liaison with interesteci organisations. The 
membership was; 

Dr R.W.J. Keay (Chairman) (tho Royal Society) 

Professor E.G. Cocking (grant-holder on behalf of the University of 

Nottingham 

Mr H.P. Dammers (Shell Research, Sittingboumo, Kent; responsible for 

processing BA Previews tapes at Shell) 

Miss Phyllis I. Edwards (ASLIB, nominated by COM) 

Dr W.M. Henderson (the Agricultural Research Council) 

Professor J.B. Jepson (current Chairman of COET) 

Dr A.K. Kent (Technical Director of UKCIS) 

Dr A.E. Needham (succeeded Dr N. Tebble as rivuit-holiirr on behalf of 

the University of Oxford) 

Dr F.H. Perring (the Institute of Biolo^r, nominated bv GOBI) 

Mr A.G.A. Pickford (the Biological Council, nominatod by GOBI) 

Dr J.E. Smith (member of the Advisory Committee? J'or Scientific and 

Technical Information (ACfJTI)) 

Dr Anne Warner (nominated by the Biological Couricil) 
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CHAPTER II 
THE DATA BASE 

A. CHARACT ERISTICS OF THE LATA BASE AND SEARCH PROCESS 

1. GENERAL 

BA Previews, the magnetic tape version of B iological Abstracts , 
consists of 36 tapes/year i.e. 3 tapes/month, two of which correspond to 
the semi-monthly publication Biological Abstracts and one to the monthly 
publication BioResearch Index. 

In order to search the BA Previews tapes it was decided to use the 
Serial Search Programs written by the United Kingdom Chemical Information 
Service (UKCIS) in KDP 9 User Code. These programs had been developed for 
searching the natural-language records of the Chemical Abstracts Service's 
data bases and, with a few modifications, could be adapted for use with 
BA Previews. For the present experiment, it was ne-essaiy to transcribe 
the BA Previews files from the IBM tapes (9 track, 800 b.p.i.) on to KDP 9 
tapes: the conversion program was written by UKCIS. Searches were run on 
the KDP 9 machine at Oxford University. 

* 

2. INFORMATION COKPAINED ON THE TAPE FILES 

The tape record contains the full bibliographic citation for each 
document (i.e. author, title, journal, volume number, date and pagination), 
together with the subject and taxonomic codes allocated by the editors, but 
not the abstract which appears in the printed Biological Abstracts. Details 
of the tape format are given in "Guide to BA Previews" 

The data elements on the tape which can be searched by the computer 
are the following: authors, natural-language terms, journal codens, CROSS 
(subject) codes, and BioSystematic (taxonomic) codes. 

(a) Authors 

Each author's name is listed, with the surname first, followed by 
the initi'^tls. 

(b) Natural-language terms 

All the words appearirif- in the title of the origin-M •'•:rticie -^re given, 
I'oUowed by 'enrichment terras', i.e. words added by the editors to m'-ike the 
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title more meaningful. The BASIC (Biological Abstracts Subjects in 
Context) index of the printed version of Biological Abstracts is compiled 
from these enriched titles. 

Details of the editorial procedures for enriching titles are given by. 
BIOSIS and the report of a study of these, which has been carried out 
during the course of the project, is summarised in this chapter. 

(c) Journal Co dens 

For all references from serial publications, the 5-character AS?M 
coden is given, followed by the journal abbreviation. 

4' 

(d) CROSS (Subject) Codes 

CROSS (£omputer Rearrangement of Subject Specialities) codes are 5-digit 
codes which are allocated to each reference by the editors to indicate the 
subject content. Each corresponds to one of the 548 subject headings which 
are listed in the CROSS Code Index 

Difficulties were frequently encountered by the liaison scientists in 
locating subjects in the CROSS index, which is based on the subject headings 
of the printed Biological Abstracts and is arranged in code number order. 
It is not truly hierarchical and there are apparent ambiguities and over- 
laps between some headings. To facilitate the use of CROSS codes, an 
alphabetical index was compiled by manual methods. This was incorporated 
in an "Index to CROSS and BioSystematic Codes""*" which was issued in 
November 1970 to each establishment participating in the project, in most 
cases for retention in the libraiy. 

CROSS codes are assigned at three levels : 

(i) Primary codes: each reference is allocated one primary code only, 
which corresponds to the printing assignment of the article, i.e. the 
subject heading under which it appears in the printed Biological Abstracts . 

(ii) Secondary codes; there may be one or more of these codes, which 
indicate other important topics mentioned in the article. 



Dee Appendix II 1 
+ An index produced by the Experimental Information Unit (8^) p'iges) 
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(iii) Tertiary uodesj these are used to indicate topics which are 
mentioned in the article, but are only of marginal interest. 

The primary and secondary codes are listed in the CROSS index of the 
printed Biological Abstracts , but tertiary codes are present on the tape 
record only. 

When the tape records were transcribed to the KDF 9, the following 
prefixes were added : 

MA J Primary codes only 

SUB Secondary codes only 

CRS Primary, secondary and tertiary codes 

In searching for a specific CROSS code, the use of the prefix CRS in 
the user profile allowed an unrestricted search to be made for articles 
allocated this code at any of the three levels of indexing. Alternatively, 
the use of the prefix MAJ restricted the search to primary codes, whilst 
use of the prefix SUB restricted the search to secondary codes. As the 
tertiary codes tend to be assigned very liberally by the BIOSIS indexers, 
user profiles often included CROSS codes with both MAJ and SUB prefixes to 
ensure the retrieval of primary and secondary codes but not tertiary codes. 

Definitions of the subject areas covered by each code are contained 

("5 ) 

in a BIOSIS in-house publication^ which was not made available to the 
project staff. V/hen new codes are introduced, or existing codes .-ire re- 
defined, a brief definition of their usage is given in the annually-produced 
"Guide to BA-Previews" 

For references where two or more topics uppeai- to be of equal import- 
ance, the primary code is assigned in accordance with the instructions 
•rontained in another in-house publication^^ \ 

A :ie tailed ctudy of CROSS code usage wa^; carried out by a member of 
♦he project ctaff in an attempt to clarify come of the problems of profile 
■jorutrutjtion v/hif. :-i arose during the project. [The renultr. of thic- atudy 
are not in':"lu'ied in the present report.] 

(e) Bio i'.y t ema tic Codes 

Bio:;y:;tematic coaer. ore "ii-digit t-iXono::.ic ?odec which --re listed in 
the Rio.Jycte':,atic O'ode Index ^ \ rju-h of the Y- Q codec represent;^ r-. sub- 
di Vic ion of the ani:::.' 1 or plant kin^do:;;, -iccord i.m' to ,:..7l.^::., ci'<.^i: or 



group of related genera. Both living and extinct forms are included, 
BIOSIS^^^ states that "The BioSystematic codes provide access to taxonomic 
references, since a taxonomic affiliation is made to ati appropriate Bio- 
Systematic code for every organise reported in a paper, whether or not the 
organism is mentioned in the title". In the present experiment, the 
prefix SYS was used to denote a BioSystematic code. An indication is given 
on the tape record when an organism is mentioned which belongs to a new or 
rearranged taxonomic group: this was given the prefix NTX. 

It 

The results of a detailed study of BioSystematic code usage are 
summarised in this chapter. 

3. PROFILE CONSTRUCTION 

The user profile consisted of a logical arrangement of one or more of 
the above-mentioned data elements^ Term weighting was \iRed both to 
simulate co-ordinate logic aid as a device to order the output. In addition 
to the simple logical operators, and, or and not, use was alao made of 
contextual logic, whereby it was possible to specify that, for retrieval 
of a reference depending upon linkage of two word terms, the second must 
occur within a specified number of words of the first. 

Table II A shows the usage made of the different data elements by the 
liaison scientists in profile construction. Nearly all the profiles (96%) 
included word terms, and most contained co-ordinations of word terms with 
either CROSS codes (86%) or BioSystematic codes (7li%). In the case of CROSS 
codes, 81% of the profiles included these at the tertiary level of indexing. 
Primary and secondary CROSS codes were used in i.0% and 32% of the profiles, 
respectively. The average number of search terms was 58 per profile, but it 
will be noted that there were wide variations among the three liaison 
scientists. Thus, the average number of search terms/profile used by 
B. Smith (77.9) was twice the number used by J. Stow (37.8). It was also 
observed that, in general. Smith's profiles used more complex logic. 
These differences reflect partly a different philosophy of profile 
construction but partly the fact that Stow's population included an unusually 
high proportion of individual university graduate students with rather 
restricted infoimation requirements. 

).. COMPUTER OUTFUT 

The computer output was produced in duplicate, the top copy being sent 
to the user and the second copy incorporated in the user's file, which was 



* 

See Appendix II. 2 
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Table ii a. profile construction: 

USAGE OF DATA ELEMENTS 





No. of profiles in which data 
element (s) present 


DATA ELEKENT(S; 


Liaison Scientist 




1 


Mrw 


BS 


! js 


■ TOTAL 




\10o)* 


(119)* 


i (119)* 

A 


(346)* 




WORD TERMS ONLY 


1 




4 


5 


1.5 


CROSS CODES ONLY 






1 

i 4 

I 


4 


1.2 


AUTHOR 


37 


20 


I 

I 30 


87 


25.1 


CODEN 


6 


3 


1 

9 


18 


5.2 


WORD TERM 


105 


119 


1 m 

i 


333 


96.2 


CROSS CODE 
(i.e. MAJ or SUB or 
CRS) 


103 


112 


\ 
1 

1 92 
1 


307 


88.7 


MAJ 


46 


78 


1 

14 


138 


39.9 


SUB 


j\j 




y 


114 


32.9 


CRS 


88 


107 


86 


281 


81.2 


BIOSYSTEMATIC CODE 


74 


103 


86 


263 


76.0 


NTX 


5 


3 


2 


10 


2.9 


WORD TERM + CROSS 


99 


112 


86 


297 


85.8 


WORD TERM + BIOSYSTEM- 
ATIC 


69 


103 


83 


25^ 


73.7 


AVERAGE NO. OF oEAfiCH 

terms/profile 


^9.8 


77.9 


37.8 


58.4 





*■ 

Figures in parenthesis give the total number of profiles which 
contributed the data. 
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maintained by the liaison scientist. 

Accompanying the first output was a profile orintout, whi.-h wns not 
issued with subsequent runs unless the profile hni been !^..iendea, i.e. -i new 
profile established. Bach output included a search terra frequen.w listing, 
which showed the number of times each search terra in the profile had produced 
a 'hit*. The number of iteras on the printout varied from 0-360 references 
per issue, the average output size being 35 references per issue. [Further 
details of output size are given in Chapter IV.] 

A specimen profile and printout is given in Appendix II. 3. A sheet 
entitled "Explanation of Biological Abstracts Output (itJTO)"* w-.s 
distributed to users in September V)10: thit: interpreted the abbreviations 
used on the printout. 

• STflblBS OF CODE ANL' ENRICHMEOT TERM ALLOCATION 

These studies, which were undertaken to provide better understanding 
of the data base, made a significant contribution to the i:nprove:iir.'r;t of 
user profiles. The initial work carried out prior to the m.'iin fe.Tuit:nent 
in September 1970 provided the liaison scientists witn guid-inoe concerning 
the allocation of codes and enrichmen-* terms. Gubseouent investigations 
were made of individual problems which arose in relation to profile 
construction. The " reports"*" of the studies of Enriehriieiit terms -md Bio- 
Systematic codes -ire summarised below. 

-I* STUDIES OF THE EDITORIAL PROCELHRES FOR AHLIMG KgYWOh:;o TO AliGMENT 
TITLES IN BA PREVIEWS" 

a) Introduction 

As the BA Previews printout contains no abiitrvvt , the user must judge 
the relevance of the source Jocuraent, in terms of hi;:; existing interests, 
from the title only. 

In an '.itternpt to I'n/.e the title more r-ieviningful , it is the accepted 
r-olicy of BIOSIS to -tdd keywords. IJo set thes-turus o+' ter-is ha? been 
compiled, so th-it rTocess involves t:.e selection of r.a t.ur': l-l-'mp-u-ige 

terrru- by the eJitoro. However, BIOJTJ^ Ogives ■•orrie ir,!"or.-:':t.ior. oon-ernirirr 
tho oategoriej of keywords which *.hey in-Jluje in their- tiMe enri-;hr::en^. 
proce.--..; : "■.eywor ii-: necess-jry to ■:!l'!rify or •i-.plify the titie ^re vd';ed 
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at the end of the title in the following approximate order : 

- physical descriptor 

- organism(s), specific name, common name 

- organ system, or tissue used or affected 

- geographical location, including geological strata 

- drug(s) and af filiation(s ) plus the word DRUG(s) linked 
to the name of a drug by a hyphen 

- important chemicals 

- specific chemical descriptor (affiliation) 

- specific enzyme(s) 

- r^pecific instr\Mentation, apparatus, or methodology 
(when new, the word NEW is inserted ahead of the term 

it modifies) 

- specific diseases 

- purpose of experimentation" 

b) Aims 

The purpose of this study was to examine some of these categories and 
determine the extent to which terms are added by the editors. It was to 
ba expected that these added keywords would increase the effectiveness of 
nattiral-language searches of BA Previews . Prom the point of view of profile 
construction, it was considered important to determine the consistency with 
which certain categories of terms are added and to ascertain if there was 
any control of synonyms. On the basis of the findii^s, it was hoped that 
some conclusions could be reached as to the degree of confidence which 
could be placed upon a significant term (e.g. hormone, drug, specific name 
of organism) being added as an enrichment, if it was not present in the title. 

For example, a user who wishes to be alerted to references concerning 
the drug "Bethanidine" can only hope for 100^ retrieval efficiency if he 
can rely upon "Bethauidine" always being added as an enrichment, even if 
it is not present in the author title. 

c) Methodoloy 

Six categories of enrichment terms were studied : 

Physical descriptors; a list of "physical descriptors" was compiled 
by scanning (a) the KWIC indexes of the enriched titles in Biological 
Abstracts (the BASIC index) and in BioResearoh Index , and (b) a 
computer-produced frequency list of enrichment terms used in six issues of 



Biologioal Abetraota and BioReeearoh Index . 

Organlam s. Geographical location and Drug names i these were 
investigated by scanning the abstracts in appropriate subject sections of 
the printed Biological Abstracts for th? presence of these categories of 
terms; their addition as enrichment was then checked in the BASIC index. 
Checks were also made for the addition of geographical locations to titles 
in BioResearch Index . 

Drug affil iations and Specific instrumentation t these were 
investigated using output from experimental profiles run against several 
issues of BA Previews (both Biological Abstracts and BioResearch Index ) . 
d) Results 

(i) Physical descriptors 

The following terms, which were used to describe publication types, 
were found added as enrichment: review, abstract, letter, symposium, book, 
lecture, note, report, thesis, and conference. 

(ii) Organismfa)} specific name, common name 

(a) Common names of organisms are consistently added when not present in 
the title, but in the abstract. 

(b) Specific names of organisms are consistently added when not in the 
title, but in the abstract. 

(c) If both the common name and the specific name are in the abstract, only 
the latter is added. 

(d) If the common name i3 in the title and the specific name is in the 
abstract, the latter is added. 

(e) If the specific name is in the title and the common name is in the 
abstract, the latter is not added. 

(iii) Geographical locations 

(a) In a sample of 250 references, which had the location mentioned in the 
title, the corresponding country was added in 22? references. 

(b) In a sample of 100 references, which had the location mentioned in the 
abstract, but not, in the title, the corresponding country was added in only 
24 references. There were some inconsistencies regarding the choice of 
enrichment terms, e.g. a reference relating to a location in Britain could 
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be enriched with Britain, British Isles, United Kingdom, England, Scotland 
or Wales, 

It was apparent that the enrichment terms were not assigned with sufficient 
consistency to provide a reliable method of retrieving articles which wore 
concernsd with specific geographical locations. 

(iv) Drugs 

In 95^ of the sample of references investigated, specific drugs cited 
in the abstract were added as enrichment. 

(v) Drug affiliations 

The affiliations of specific drugs in the title or abstract, e.g. HORMONE- 
DRUG, appear to be consistently added, if the hormones mentioned are used as 
drugs in order to observe their effect on a particular system. 

(vi) Specific instrumentation 

The subject of "electron microscopy" was studied. It was found that 
this term is not always added to titles of references concerning this topic, 
even if the term is present in the abstract. Synonyms or related terms such 
as "electron beam microscopy", "electron micrograph", "fine structure" and 
"ultrastructure" are also used and these are not necessarily identical to 
the term appearing in the abstract. 

(vii) Importance of enrichment terms in searching BA Previews 

In a random sample of 100 references, which included some with non- 
enriched titles, it was found that an average of 3.1 words or phrases were 
added to the author title. 

In a sample of 150 references, all of which had been retrieved by word terms 
only, it was found that 31^ would not have been retrieved wilhout the presence 
of the enrichment terms. 

2. STUDY OF BIOSYSTEMATIC CODE USAGE 
a) Methodology 

Studies of the allocation of BioSystematic codes made use of specially- 
constructed profiles which v,ere run against one or more issues of BA Previews . 
The output from these experimental profiles included a printout of the codes 
allocated by BIOSIS to each paper. For scrie aspects of the study, :-t was 
also necessary to scfin the correu ponding abstracts in the printed 
Biologic-il Abstracts but, with one exception, reference wf;3 not m.'ide to tha 
original article. 
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(i) -:o.;ir^: o::;.:jiGr.J 

In urti';le:: where an organism is mentioned in the enriched title, 
and/cr the uppropriuie taxonomic CR03C' code is allocated, the appropriate 
Bio3y3tematic code Hllocated almost invariably, 

(ii) Coding erroro 

The probability of a BioSystematio code for p*:rticulrjr t:iX'ononiic 
group retrieving an article unrelated to that ^^roup is nerli^Tihle. 

(iii) :jix studies of BioSystematic code allocation 

The allocation of Biooysteiaa tic codes was studied in the following: 
six fields of applied biology : 

Food Technology ; if the type of food involved ir. -mentioned in 
the enriched title, there is a hign probnbility that the HioJystema tic code 
corresponding to the organism acting as the source oV food will be vllocated. 

Forestry and forest products : the appropriate Hi o3y::. term tie code 
was always allocated when specific genera or species *. sre me*tioned. 

Plant-derived drugs : all but two of Yf articles retrieved were 
indexed with the appropriate BioSystematic code for the plant irorn which 
the drug was derived • 

Mycology and antifungal a^>:ents : all but f ive of 40 ref or ericei3 
allocated the -IHO'cVJ codes for mycology (cR33^-00'0 '^nd ^jntifungal a^rents 
(CR.>:;8";08) v/ere alno allocated a Biooynte-riatic voic for fia.f''i. 

?is.;ue culture ; all but two of U. reference^ -^i J located the CliOol'j 
code for tis..:ue culture (CRo^'-'^^OO) were 'jIso 'illo:*:te^J the Hiooyr-te:na tic 
code for the -.nl-ia] or plant frori. whi-h the cell..' were orir''inal]y obtained, 

Clinic'l Tiedici; e ; -ill but tv;o of 11.-- ref efer:.:e- 'd located one of 
five 'ROo.' codes for clinical .-nedicine were '.illoc^ted the ido.Jye.terfia tic code 
for Ko:riini aae {^^Y^jSc I ). 

(iv) .• •ope of ^rener'i 1 Hiooyote:r;':ti >:;o.ie:: 

A v'tviy v;a:- Traue of tnree ^reneral co;e.^ i.e. Xi-roor.^-.r.i.;-:.* (.:y.:01000) > 
viiOr ia t*.i (/Y.>.v 000) ani Ver"etr*itu (oY.>i;i.O), \o 'ert*: ir. w.'.'r* r.^-r r,ey 
were ^..^eigne: or,!;/ to arUcleK^ v;:ii-':; either -le-r/ribe i^-er.er-jl prir.': i pier or 
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fail to mention specific organisms* It was found that leG out of 
references did not mention specific organisms; in the remaining ^7 
references one or more species were mentioned, though general principles 
were also discussed in half of these. 

It was difficult to determine the policy underlying the distinction 
between Chordata and Vertebrata. However, it was found that no references 
were allocated both codes. 

(v) Overlap between CROSS codes and BioSystematic codes 

Microorganisms ! analysis of 91 references allocated CRS29500 
and/or SYSOIOOO indicated that the CROSS code was uaed when a paper dealt 
with a fundamental activity of microorganisms, whilst the BioSystomatic 
code was used when microorganisms (often unspecified) were relevant to some 
aspect of the paper, though in many cases this was merely inferred, e.g. 
infectious diseases* 

Paleontology : analysis of printouts of 157 articles retrieved 
by a profile composed of the three CROSS codes and the three BioSystematic 
codes for Paleontology showed that, in all but 16 cases, the appropriate 
BioSystematic code was accompanied by the equivalent CiiOSS code, e.g. 
3Y351000 (Paleozoology) was assigned together with CRS63000 (paleozoology). 
The general code SYS27000 (Paleobiology) was only assigned to four papers. 
The general code CRS64^00 (Paleobiology) was used for general topics such 
as stratigraphy and analytical techniques. 

(vi) Some recommendations for using BioSystematic codes 

In profile construction, the decision whether to use specific names or 
the appropriate BioSystematic codes will depend upon the number of 
organisms to be included and the amount of information required about them* 
The following recommendations are offered: 

(a) When broad categories of organisms are required (e.g. Canidae) it 
is advisable to use BiooystemiJ tic codes. 

(b) When only one or a few species or genera are required, it is preferable 
to Hot cpecific, generic and common names in the profile, rather than use 
the appropriate BioSystem^i tic code, which would result in the retrieval of 
much unwanted material. 
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(c) When only Homo sapiens is of interest, it is preferable to use 
SYS86215 (Hominidae) instead of listing all the possible synonyms for 
•human* (e.g. man, adolescent, girl). 

(d) Analysis of a printout of references relating to methods, techniques 
and theoretical models showed that the general BioSystematic code (e.g. 
Chordata, Vertebra ta) was sometimes used in preference to a specific code 
(e.g. Hominidae). Thus, either these general codes should be included in 
such profiles or, preferably, word terms describing specific techniques 
should be listed. 
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CHAPTER III 
POPUUTION OF USERS 

SELECTION OF PARTICIPANTS FOR THE EXPERIMENT 

1. DEVELOPMENT PHA.3E OF THE EXPERIMENT (JANUARY - MARCH 1970) 

A pilot group of 55 biologists was recruited from the host universities 
of Oxford and Nottingham to enable the liaison scientists to acquire practical 
experience in techniques of interviewing and of constructing user profiles. 
The two grant-holders invited a number of biologists to seminars, at which 
the aims of the experiment were described, so that volunteers could be 
acquired who were willing to act as test users. 



TABLE III A. PILOT STUDY PARTICIPANTS 



Iftiiversity 


Liaison Sciercist 


I 

i 

Departments 


No* of test 
users 


Oxford 


M. Williams 


Zoology i 


16 


Oxford 


B. Smith 


Physiology ) 
Biochemistry, ) 


11 


Nottingham 


J. Stow 


Botany ) 
Zoology ) 


28 


TOTAL 






55 



2. OPERATIONAL PHASE OF THE EXPERIMENT (APRIL 1970 - APRIL 1972) 
a) Main recruitment 

The main recruitment commenced in September 1970. As users of 
biological literature include workers in widely differing subject areas {e.g. 
plant taxonomy, genetics, immunology), it was appreciated that their 
literature needs would vary considerably. It was also expected that the 
needs of users in different types of employment would vary. An attempt was 
therefore made to recruit a population which was as representative as possible 
of biologists in the United Kingdom* 
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No up-to-date published statistics were available but, using data 
obtained in a government survey^ it was possible to obtain estimates of the 
percentage of workers in different aubject fields related to their type 
of employment. These figures were used as a guide to the selection of a 
reasonably representative sample of biological research workers in the 
United Kingdom. The choice of the university departments, and government 
and industrial research organisations was also influenced by the practical 
consideration of proximity to Oxford or Nottingham. 

As a result of the publicity given to the project by an announcement 
in OSTI Newsletter (December 1969)^''^^and an article in Nature (l3 March 
1970)^*^^) a considerable number of enquiries had been received from both 
research workers and information officers. Wherever possible, their 
institutions were included, particularly if they belonged to under-represented 
subject fields. 

The majority of the academic population (75^) was recruited from the 
host universities of Oxford and Nottingham. In the case of the government 
workers, the main recruitment was from the laboratories of the Ministry of 
Agriculture, Fisheries and Pood and the Agricultural Research Council (ARC). 
The number of biologists (66) interviewed at ARC establishments was in fact 
considerably higher than the required figure. However, it was considered 
valuable to include these, as the ARC intended to make an independent 
assessment of the value of computer search services to their organisation. 
Two of the Medical Research Council Units contacted agreed to participate 
in the experiment. None of the three research associations which were 
approached wished to join in the experiment. 

Eight information centres, which included three Commonwealth 
Agricultural Bureaux and three Specialised Information Centres, also took 
part in the experiment. In order to allow for the broad subject coverage 
of the centres, up to three experimental profiles were allocated to each 
centre. 

In an attempt to include all types of users of biological literature, 
some research workers in independent research establishments, and four 

The data collected in this survey was subsequently incorporated in the 
CSP Biology Manpower Survey* 1972 (Cmnd ii737)t 
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amateur naturalists, were also recruited. 

An additional 16 users were recruited by information officers in 
some of the establishment j visited. Most of the responsibility for profile 
construction and amendment was undertaken by them. 

At the completion of the recruitment, a total number of 337 biologists, 
in 60 establishments, had been interviewed by the liaison scientists and 
392 profiles had been written. 

Table III B shows the number of types of establishments visited, the 
users recruited and the profiles written for each category of employment. 
Table III C shows the number of profiles written in specific subject 
categories. A list of the 60 establishments visited is given in the Appendix? 

b) Liaison with users 

The experimental procedures for liaison with the user, which were 
developed during the pilot studies, were adopted for the main recruitment 
stage of the experiment. 

With the exception of the host universities, wherever possible, the 
invitation to participate in the experiment was addressed to the librarian 
or information officer of the establishment to be visited. An advantage 
of this approach was that the library usually organised an initial seminar 
and arranged an interview time-table. Furthermore, it was considered 
important to involve the library/information department in the provision 
of mechanised information services such as BA Previews. 

On the first visit of the liaison scientist to the institution (e.g. 
university department, government research labomtory) an introductory 
talk was usually given to a group of potential participants. The aims of 
the project, the chara<-teristics of the data base, and the feedb-.ck expected 
••ro;.'i the user, were '-!es-ribed, with the aid of a handout^'. IndivLdu-ii 
Interview:: were then urruiged for research workers who were interested in 
Ui<in^. p'lrt in the pro.ie'.'t. 

The rir.-t interview (l-i;. hours) of the li'iison scientist with the 
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TAB^K III C. 



- % - 

PROFILES BY SPECIFIC SUBJECT CATEGORY 



The figures in brackets are the numbers (or percentages) of profiles 
estimated for each subject category from the data obtained in a 
government survey of the employment of biologists. 



Subject Category 


No. of 


Profiles 




— 

1. 


Agricultural Science 


23 


(48) 


5.9 


(12.0) 


2. 


Aquatic Biology 


10 


(8) 


2.6 


(2.0) 


3. 


Biochemistry 


36 


(40) 


9.2 


(10.0) I 


4. 


Biometrics 


1 


(6) 


0.3 


(1.5) 




Biophysics 


16 


(18) 


4.1 


(4.5) 


! 6. 


Cell Biology 


15 


(8) 


3.8 


(2.0) 




Ecology and Conservation 


24 


(16) 


6.1 


(4.0) 


8. 


Embryology 


2 


(6) 


0.5 


(1.3) 


, 9. 


Endocrinology 


12 


(6) 


3.1 


(1.5) 


10. 


Entomology 


17 


(6) 


4.3 


(1.5) 


11. 


Experimental Psychology 


5 


(10) ! 


1.3 


(2.5) 


; 12. 


Food Science 


10 


(12) i 


2. 6 


(3.0) 




Forestry 


5 


(6) i 


1.3 


(1.5) 


! 14. 

1 


Genetics 


11 


(12) i 


2.8 


(3.0) 


1 15. 


Immunology 


15 


(8) 


3.8 


(2.0) 


1 16. 


Medicine 


5 


(40) 


1.3 


(10.0) 


17. 


Microbiology 


18 


(24) 


4. 6 


(6.0) 


I 18. 


Nutrition 


1 


(2) 


0. 3 


(0.5) 


; 19. 


Paleontology 


5 


(4) 




(l.O) 


; 20. 


Parasitology 


9 


(4) 


2. 3 


(l.O) 


21. 


Pathology and Forensic Science 


9 


(10) 


2. 3 


(2.5) 


22. 


Pharmacology and Toxicology 


23 


(22) 


: .5*^ 


(5.5) 


; 23. 


Physiology 


35 


(28) 


i 8.9 


(6.5) 


^ 24. 


Plant Pathology 


19 


(10) 


4.8 


(2.5) 


•; 25. 


Plant Physiology 


32 


(28) 


8.2 


(7.0) 


26. 


Public Health 


1 


(2) 


'■ 0.3 


(0.5) 


27. 


Systematic Botany 


7 

1 


(A) 


1.8 


(i.o) 


28. 


Systematic Zoology 


10 


(4) 


1 2.6 


(l.O) 


29. 


Veterinary Science 


5 


(4) 


! 1.3 


(l.O) 


30. 


Virology 


12 


(6) 


3.1 


(1.5) 




Total 


392 (400) 


100 


100 
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user started with the completion of a questionnaire* designed to ascertain 
current literature searching practice and attitude to Biological Abstracts . 
The user was then asked to define his research topic so that a narrative 
♦Statement of Requirements' could be written ^' A lengthy discussion ensued 
between the liaison scientist and the user, who was encouraged to give a 
more detailed description of his research interests, pointing out which 
topics he wished to be included and any aspects not required. On the 
basis of the information acquired, the liaison scientist proceeded to 
compile a user profile. 

c) Profile construction 

The User Profile consisted of a logical co-ordination of terms or 
codes from one or more of the following categories*. 

(i) Words 

(a) appearing in titles of relevant articles 

(b) added by BIOSIS to enrich titles 

(ii) CROSS Index (subject) codes 
(iii) BioSystematic (taxonomic) codes 
(iv) Authors 
(v) Journal codens 

In computer searching, the terms (or codes) in the profile are matched 
against the record on the BA Previews tape and a printout of relevant 
references is produced. 

Most of the participants were offered only one BA Previews profile, 
which attempted in some cases to cover the entire range of their interests, 
and in others to define one specific area of interest. Some users were 
allowed to have more than one profile, for one of the following reasons? 

(i) The user wanted separate printouts for two or more unrelated 

topics 

(ii) It was impossible to express adequately all the subject 

requirements of the user within the logical capabilities of 
one profile. 

* 

See Appendix II I. 3 
* A saaple of users' requirements are given in Appendix III J 3 
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Conversely, in order to economise in interviewing time and the number 
of profiles acquired, 3^ Group Profiles were written, whereby more than one 
person attenueJ the interview and contributed to the construction of the 
profile. 

A further 12 Shared Profiles were written, whereby one person only 
was interviewed, acting on behalf of one or more others. (The sharing of 
a printout with a research student was not included in this category). 
Some profiles which were written for one user only were subsecuently found 
to be shared with colleagues. Details of^Group and Shared profiles are 
tabulated in an Appendix . 

A few weeks after the first interview, the user received a printout, 
which corresponded either to the serai-monthly issue of Biological Abstracts 
or to the monthly issue of BioResearch Index . 

d) Feedback from users 

After the users had received a few initial printouts, a second interview 
was arranged to discuss the effectiveness of the search and to make any 
necessary amendments to the profile. 

Accompanying each printout was a Feedback card *^. This provided an 
e.sr-ential means of rapport between the liaison scientist and the user. As 
well a3 marking the number of references which were either "Important" or 
"Relevant" to him, the user was encouraged to write comments on the service 
ana return the card to the liaison scientist. Regular examination of these 
cards not only provided the liaison scientists with an indication of how 
the profile was performing but also alerted them to omissions and errors 
which might otherwise have been overlooked. 

To assist users in making a relevance judgement, the following 
aefi nit ions were given; 

IMPORTANT ; A reference directly related to your research interests, which 
you will read uts soon -is possible, i.e. it is essential that you know of it. 



Jee Ar-nen iix III .4 
* Jee Arpen-iix III.- 
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RELEVANT ? A reference which, though not important, is of some interest to 
you, i.e. you are glad that you were notified of it. 

The IMPORTANT category was intended to be interpreted restrictively 
and the RELEVANT category generously, and the judgement to be independent 
of whether a reference was already known to the user. 

This gave rise to problems for many users, who misunderstood the 
distinction between these two categories and included the IMPORTANT 
references in the total number of RELEVANT references. In order to avoid 
this, it was decided in October 1970 to rename the IMPORTANT and RELEVANT 
categories as MAJOR VALUE and MINOR VALUE, respectively^and to redefine them 
as follows; 

MAJOR VALUE ; Very centrally related to research interests and could make 
a significant contribution. 

MINOR VALUE ; Of interest, and happy to have been informed of it, but 
of no great loss to have missed it. 

Subsequent discussions with the users revealed that the renaming of 
the relevance categories eliminated any misinterpretations. 

An analysis was made of the comments on the feedback cards returned 
by a sample of 100 users; the resur*:s are given in an Appendix . These 
results can be summarised as follows: 

It was found that 88^ of the sample made comments on their feedback 
cards, ranging from 1 to 36 cards/user. Prom the point of view of the 
liaison scientists, the most valuable comments were those relating to 
profile construction. Users appeared to find it more easy to comment upon 
unwanted subjects than suggest additions to their profile. Twenty two of 
the 100 users remarked that their printout included unwanted topics, whilst 

were able to specify terms, codes or journals which should be deleted 
from the profile. However, only six users suggested additions to their 
profile. Users also drew attention to spelling errors in their profile and 
cases of the truncation of word terms causing the retrieval of irrelevant 
references. After the amendment of their profile, U3eri3 commented upon 

# 

See Appendix III. 5 
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its efficacy, expressing either general satisfaction (l8), dissatisfaction 
(4) or indifference (l). General comments were also made upon the efficacy 
or otherwise of the profile in covering their subject interests. 

Some interesting comments were made on t^e value of the references, 
Ih users remarking that some MAJOR VALUE references were from joumals 
they did not scan, or had missed in their searching. There were also 

comments on the non-availability of references. Three users had been unable to 
obtain references from the NLL. Difficulty was sometimes encountered in 
making relevance decisions, e.g. distinguishing between MAJOR and MINOR VALUE 
(4), judging relevance from the title only (2). Two users would have liked 
a third relevance category to include articles of "fringe interest". Sixteen 
users made adverse comments on the illegibility of the printout and six could 
not understand the abbreviations. When a better quality printout was 
produced, six users commented favourably on this. 

®) Quantitative evaluatio n phase of the experiment 

in order to calculate parameters such as precision, recall and coverage 
(See Chapter iv) it was necessary, over a limited period, to obtain more 
detailed information than that given on the feedback cards. However, it was 
appreciated that many users would not be prepared to spare the time to provide 
further information. Accordingly, on 5 May 1971, a letter* was sent to all 
users. With a description of how much work would be entailed. They were 
given three options; 

(1) To take part in the quantitative evaluation phase 

(2) Not to take part in the quantitative evaluation phase, but 
to continue to return feedback cards 

(5) To withdraw from the experiment. 

At the start of the evaluation phase (June 1971) there were 302 users 
(346 profiles) of whom 151 users (I6O profiles) had agreed to be 'evalueest. 
Nineteen of these »evaluee' profiles were Group Profiles (See page 24). 
Although the evaluees were self-selected on the basis of willingness to 
co-operite, they proved to be reasonably representative of the subject and 
employment categories of the total population of users. The employment and 

* 

See Appendix I I I. 7 
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subject categories of the evaluees are 'inown in tv;o tables in the Appendix • 

Users opting to withdraw from the experiment at this stage brought the 
total number of profiles deleted since the start of the project to 59. 
However > eight of these transferred their profile to a colleague, 

B. CHARACTERISTICS OF PARTICIPANTS IN THE EXPERIMENT, WITH SPECIAL 
RK ERENCE TO THEDl LITERATURE SEARCHING HABITS 

1. GENERAL 

Reference has already been made to a questionnaire^ completed at the 
first interview. This was designed to ascertain the existing literature 
searching habits of the biologists participating in the experiment. It was 
intended for completion by the liaison scientist in discussion with the 
user. Of the 353 biologists interviewed^ 333 provided a completed 
questionnaire. 

No. of users interviewed by liaison scientists up 
to March 1961 

(Questionnaire completed) = 333 

No. of users interviewed by liaison scientists up 
to March 1971 

(No questionnaire completed) = 4 

No. of users interviewed by information officers 
up to March 1971 

(Questionnaire not completed) = 16 

Total no. of u^ers interviewed ^ 353 (392 profiles) 
No. of users who withdrew prior to 30 June 1971 - 59 

No. of users who withdrew from experiment, but 
transferred their profile to a colleague = 8 

Total no. of users at 30 June 1971 = 302 (346 profiles) 

Participants were asked to indicate on the questionnaire the length (in years) 
of their research experience. Replies varied from "1 to 1.5 years tut the 
largest group (32%) were those in the 0-5 year range. 



* 

See Appendix III. 8 and 111,9 
"** See Appendix III. 3 
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2. USAGE OF THE LITERATURE 

a) Manual literature searching 
(i) Primary ^journal usa^ e 

Users were asked to list up to 10 primary journals which they scanned 
regularly. In all, 698 journals were cited by one or more users. 

As might be expected, the most frequently cited journal was Nature 
(159 users) , the next three most frequently cited journals were biochemical 
journals: Biochemica l Journal (72 users), Biochimica et Biophvsica Acta 
(51 users) and Journal of Biological Chemistry (47 users). A table in the 
Appendix lists the primary journals w>iich were cited by 10 or more users. 

(ii) Secondary journal usage 

A list was given of 10 common secondary journals, and users were 
asked to indicate which of these they used for current awareness and/or 
retrospective searching. Their answers are shown in Table III D. It is 

apparent that Current Contents was most widely used for current 
awareness purposes and Biological Abstracts for retrospective searching. 

In addition, they were asked to cite any specialised secondary 
journals (other than those shown in Table III d) which they consulted 
regularly. One hundred and fifty different titles were cited, of which 
only 55 were quoted by more than one user. Table III E lists "those cited 
more than 10 times. With the exception of Bibliography of Reproduction , 
they are all Commonwealth A^icultural Bureau.: publications. 



Discussion with users indicated that Nature would have been quoted by 
aiore, but was not considered to merit inclusion, being classed by them 
as a journal of general interest, rather than a core journa] 

See Appendix 111,10 
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TABLE III E. MAMUAL LITERATURE SEARCHING: 
SECONDARY JOURNALS CITED BY 


SPECIALISED 
10 OR MORE USERS 


Title 


No. 


of Citations 


Veterinary Bulletin 




26 


Horticultural Abstracts 




25 


Review of Applied Entomology ) 
Soils and Fertilisers ) 




18 


Helminthological Abstracts 




14 


Field Crop Abstracts 




.12 ! 


Weed Abstracts 




11 1 

1 


Bibliography of Reproduction ) 
Nutrition Abstracts and Reviews ) 
Review of Plant Prthology ) 




10 ! 

t 

1 



(iii) General approach to manual literature searching 

When asked if they considered their literature searching was adequate, 
the majority (258 users: 71^) replied "No". [However, it was apparent 
from comments made at second interviews that some of the 81 users (24?^) 
who had replied fes" to this question, had subsequently come' to realise 
that they had not been covering the literature as comprehensively as they 
originally thought.] 

The reasons given by those who said "No" are categorised in Table III P. 
Hechanised information services 

Users were asked if they had previous experience of a mechanised 
service. Eighty four users replied "Yes" and mentioned the services shown 
in Table III G. A MEDLARS one-off retrospective search was the service 
cited by the largest number of users. 
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TABLE III INADEQUACy OF MANUAL LITEBATUHE SEARCHING 



Reason 



No. of times cited 



1. Lack of time 



2. Diverse interests • too many relevant 
journals to scar 

3» Lack of adequate library facilities 

4. Language barrier: lack of adeauate 
translation facilities 



5. 



6. 
7- 

8. 

10. 



Lack of a systematic searching policy 

A general feeling of inadequacy 

Inadequate journal coverage (including 
poor currency, bad format) 

Only a limited number of journals scanned, 
as a general policy 

No literature searching done: rely entirely 
upon personal contacts 

Lack of relevant journals, since user is 
only worker in his field 

Note : Some users gave more than one reason 



145 

47 

46 

29 

22 
18 

16 
13 
10 

3 



TABLE III G. MECHANISED INFORMATION SERVICES 



Mechanised Information Service 



No. of times cited 



(i) 


MEDLARS (Retrospective) 


44 


(ii) 


MEDURS (Monthly) 


5 


(iii) 


A3CA 


12 


(iv) 


CT 


7 




CBAC 


17 




CAC 


3 


(v) 


Ringdoc 


13 




Agdoc 


1 


(vi) 


Drugdoc 


1 


(vii) 


Inspec 


1 
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c) Non-users of Biological Abstracts 

One hundred and fifty five (46!^) of those interviewed did not use the 
printed Biological Abstracts . They were asked to give one or more reasons: 
their replies are shown in Table III H. 



TABLE III H. BIOLOGICAL ABSTRACTS: REASONS FOR 
NON-USAGE OF BIOLOGICAL ABSTRACTS 


I 

1 

I 

1 


Reason 


No. of times cited 




No pubscription by the organisation 


68 


(ii) 


Inaccessibility 


29 ; 


". (iii) 


Do not like format of indexes 


14 


(iv) 


Do not consider journal coverage 
adequate for own subject field 


9 


: 


Rely on another service 


60 


(vi) 


Do not find any need for abstract 
services 


8 


(vii) 


Tend to rely on "Invisible Colleges" 


48 


(viii) 


Other 


i 

20 



The most coininonly cited alternative information service was an 
internal Library Information Bulletin (ll users), followed by 
Current Contents (s users) and Chemical Abstracts (? users). 



d) Users of Biologioal Abstracts 

The 178 users {SWQ who did use the printed Biologioal Abstract s were 
then asked to state the type of searches they made of it, including the use 
they made of the indexes, and to give their opinions on certain aspects of 
the service* Their replies were as follows: 

(i) Current Awareness searches 

Biological Abstracts v^s used for current awareness pxxrposes by 
105 users {S9% ol BA users; }2% of the full experimental population of 
333 users*)* However, only 33 of these 105 users claimed regular usage; 



ERIC 



i»e# 6Q% of the population did not use BA for current awareness purposes 
O (of. p* 8ij)* 
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the remaining 72 used BA only spasmodically. 

(ii) Retrospective searches 

One hundred and sixty five participants (92% of BA users; 50?^ of all users) 
used Biological Abstracts for retrospective searching, 36 frequently and 1 29 
infrequently*. 

(iii) Usage of the indexes 

Usage of the indexes to Biological Abstracts is shown in Table III I, 

TABLE III I. USAGE OP BIOLOGICAL ABSTRACTS: INDEXES 

Curient Awareness Retrospective 



Index 


Searching 


Searching 


Aut}i'->r 


hh 


ih2%)* 


86 


(52%)* 


BASIC 


51 




71 


(») 


CROSS 


3 


(3%) 


8 


(5%) 


BioSystematic 


21 


(20%) 


25 


(15%) 


Appropriate subject section 


71 


(68%) 


105 





Pigxires in brackets are percentages of those who use Biologic al 
Abstracts t i*e* 105 (Current awareness) and 165 (RetrospectiveT* 



The most popular method of searching Biological Abstracts was item-by- 
item scanning of selected subject sections. This method was adopted by 68% 
of those who used the journal for current awareness searches and by 6k% of 
those who used it for retrospective searches. Though the majority also made 
some use of at least one of the four indexes (author, BASIC, BioSystematic, 
cross) there was evidence of substantial under-use of the printed indexes. 
The BASIC index, which might have been expected to be used extensively for 
current awareness purposes, was in fact used in only k9% of current awareness 



Users • replies to this question can be compared with replies to Question 
2(a) (ii) regarding secondary journal usage (Table III D). In answer to 
the latter question only 95 said they used Biological Abstracts for current 
awareness purposes and 152 that they used it for retrospective searches. 
It seems that some users, though not considering that Biological Abstracts 
justified inclusion in their list of secondary jotxmals, when asked a direct 
question, said that they used it spasmodically or infrequently. 



Authors note: this figure was wrongly quoted as hS% in the synopsis of 
this report (OSTI Report No. 5^39)* 



searches. Only ten participants (6% of those who used Biological Abstracts ) 
claimed to make use of the CROSS index and usa^e of the BioSystematic index 
(20^ of users) was also lower than expected. In discussion with the 
participants, the impression was gained that a significant number of biologists 
either disliked the printed indexes or did not understand how they should be 
used. As a result they sometimes found it diffic\ilt to establish an effective 
method of searching and failed to locate potentially interesting articles. 

In contrast, nearly all search profiles for BA Previews made use of 
CBOSS and BioSystematic codes, which in most cases were linked with other 
types of index terms, e.g. title words (see Chapter II ). Such coordinations 
are easily accommodated in a computer search, but the equivalent manusd 
search would be extremely laborious. 

(iv) Journal coverage 

When asked if they considered that the journal coverage of Biological 
Abstracts was adequate for their research interests, their replies were as 
follows: 



Yes 



11.2 



19 



(80%) 
(11%) 



Bo not know 
No reply 



] 17 (10%) 



The interests of those replying "No" covered physical as well as 
biological sciences, but in addition there were seven core biologists, 
mednly ecologists, who considered the coverage inadequate. 



(v) Quality of the abstracts 

Users were next asked if they considered that the abstracts were 
sufficiently informative: their replies were as follows: 



- hi - 



Yes 161 (90^) 

No 15 

Do not know ) . 
No reply ) ^ (1%) 

(vi) Time-lag 

Finally, users were asked if the time-lag between publication of an 
article in a primary journal and its appearance in Biological Abstracts 
rendered the latter inadequate as a current awareness tool. Their replies 
were as follows: 

Yes 95 (52^) 

No 81 (46^) 

Do not know ) . 
No reply ) 

3. USERS WHO WITHDREW FROM THE EXPERIMENT 



i2fo) 



Mention was made on page 27 of 59 users who withdrew from the 
experiment between February and June, 1971 and whose profiles were deleted 
(51 ) or passed on to a colleague (s). 

During this period, 10 users withdrew because they were leaving the 
U.K. for work abroad. The remaining 49 were deleted in two batches, as 
follows: 

(a) Pebruaiy - March 1971: 11 users, who had failed to return 
feedback cards regularly and did not reply to a written 
reminder . 

(b) May - June 1971: 38 users, who in response to the T'^tter'*' of 
5 May (See page 38), opted to withdraw at the start of the 
quantitative evaluation phase. 

Fourteen of the users who withdrew had been recruited in the pilot 
studies. A total of 53 academic users withdrew, compared with nine 



See Appendix III. 11 
^ See Appendix III. 7 
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government, 14 industrial, and two from independent research establis^^en-ts 
Twenty four of the 30 subject categories were represented, no one particular 
subject area having an excessive number of dissatisfied users. 

In order to ascertain why users withdrew fromthe experiment, they 
were sent a questionnaire', on which they were asked to indicate their 
reasons. These are shown in Table III j. 



TABLE III J. REASONS FOR WITHDRAWAL PROM THE BA PREVIEWS 
EXPERIMEM-: REPLIES FROM 46 DELETED USERS 



1. The BA Previews service has proved of no value 

(a) Have already seen most of the references 

(b) The references are too old to be of any use 

(c) The coverage of BA Previews is inadeauate 

(d) The printout includes too many irrelevant 
references 

2. Rely on another service 

(a) Abstracting or indexing journal 

(b) Library/inf ormation service 

(c) Invisible College 

(d) ASCA profile 

3. Analysis of printout and return of feedback is 
too time-consuming 

4. Format of printout is unacceptable 

Research topic has changed, rendering profile 
inadequate 

6. Leaving the U.K. for work abroad 

7. Other 



4 ) 
4 ) 
4 ) 

10 ) 



7 ) 
•8 ) 

? ) 
1 ) 

13 

4 

4 
10 



22 



18 
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CHAPTER IV 
SEBYICE PEHPOBMANCE 

1. imODUCTIQN 

The followiiig parameters were selected as the most suitable quantitative 
measures of the performance of the BA Previews S.D.I, service: 

(1) Precision: the proportion of references retrieved by the BA Previews 

service which the user Judges to be relevant 

(2) Output sizet the average number of references retrieved from eacxx 

' issue » of the BA Previews data base (there are three 
» issues* of BA Previews each month) 

(3) Recall: the proportion of relevant references in the data base 

which are retrieved by the service 

(h) Coverage: the proportion of relevant references which are covered 

by the data base 

(5) Overall recall: the proportion of relevant references which are retrieved 

(irrespective of whether or not they are covered by the 
data base) 

(6) Novelty ratio : the proportion of relevant references retrieved by the 

service which were not previously known to the user 

(7) Relevance the extent to which the docmnent surrogate provided by 

predi ctability: / x 

^ the service (the enriched title) allows the user to make 

a correct prediction of the relevance of the actual 

document 

Formal definitions of each of these parameters are given in subsequent sections 
of this chapter. 

One hundred and fil'ty one users, representing I60 profiles, agreed to take part 
in the evaluation (cf. p. 38), but some evaluees did not return all the required 
feedback and, in the case of the recall and coverage samples, we did not analyse 
all the data which was retumed. The number of users contributing iata to the 
above performance parameters varies, but is always less than the maximiim of 151 
users . 

2. RELEVANCE DECISIONS 

Estimates of performance parameters for each user, with the exception of 
output size, are derived from the user's relevance decisions on individual 
references. The majority of relevance decisions were based on a document 
Q surrogate rather than on the actu£tl document. In the case of the precision 
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samples, users were asked to decide the relevance of an article from 
information provided in the computer printout (primarily the enriched title). 

It is recognised that the ultimate aim of a retrieval system is to 
alert the research workers to relevant documents and relevance decisions on 
the actual documents must usually be considered as definitive. Nevertheless, 
relevance judgements on the surrogate are considered to be an acceptable 
basis for estimates of performance ratios. Such judgements correspond to a 
specific stage in the information process and have real significance to the 
user. Thus, the user does not refer to all the documents represented in his 
output but decides, on the basis of the document representation offered by 
the service, whether further action is justified, e.g. reading the document 
or noting the reference for future use. Furthermore, routine monitoring of 
profile performance and the amendment of the search strategy are based on 
assessments of document surrogates . 

Some relevance decisions based on the full dociunents were obtained from 
each user in order to calculate relevance predictability for the surrogate 
provided in the computer printout from the BA Previews service. These data 
allowed comparisons to be made of precision ratios derived from the relevance 
decisions based on the full document and on the enriched title respectively. 

Users were asked to classify references as MAJOR VALUE, MINOR VALUE or 
IRRELEVANT. Definitions of MAJOR and MINOR VALUE categories are given in 
Chapter III (page 37). 

The definitions of performance ratios (recall, precision, coverage and 
novelty) are expressed in terms of relevant references (relevant = MAJOR VALUE 
+ MINOR value) but similar parameters can be defined for either MAJOR or 
MINOR VALUE references. Estimates for relevant references are given for all 
parameters and, in most cases, the corresponding value for MAJOR VALUE items 
only was also calculated. Values for MINOR VALUE items alone are given only 
for the recall and coverage ratios. 

3. SAMPLING METHODS 

Values or performance r^itios must be estimated from reference samples . 
For example, it is not feasible to identify all relevant references in the 



A more detailed discussion of relevance decisions and other aspects of 
evaluation methodology is given in reference (9), 



- 51 - 

data baae in order to compute a recall ratio • A reasonably simple method 
must be devised for obtaining a sample of relevant references (the recall 
sample) which is random with respect to the probability of the reference 
being retrieved by the profile • 

In order to estimate precision ratios, use can be made of a chronological 
sampia consisting of the output from one or more issues of the data base. 
Samplirxg methods for other parameters will be discussed in the appropriate 
sections. 

STATISTICAL KETHODS ^^^^^^^^^ 

W/.^r* the exception of output size all the perfoimance measures are 
ratios:. Xean values for a population of users can therefore be obtained 
eiti;*^r iy averaging ratios or by averaging numbers. In the foimer method a 
rati: ^.i; conputerl for each user and these ratios are then averaged. In 
contrac t; a number aveicige is obtained by summing the numerators and de- 
nomiTiators separately and then calculating the quotient of the two sums. 
Thus, for a ratio of the general foim ^/b for a population of N users: 

ratio average = ^ X ^ ; niunber average = ^4 ^ 

N 

The ratio average is \xsually preferred as it treats the individual lAser (or 
profile) as the sample imit and each user is equally weighted in the 
calculation of the mean value. If a number average is usedi a user who 
contributes a Very large reference sample will have a disproportionate 
influence on the mean value. If reference samples are of equal size for all 
users the two methods given identical values. 

Unless otherwise indicated^ all estimates of mean values reported in 
this chapter were obtained by averaging ratios. The only major exception is 
the relevance predictability data given in Table IV D. As the sample sizes 
for this data varied only slightly (7 - 12 references), the number averages 
are unlikely to differ significantly from ratio averages. 

For some * groups or •shared* profiles (cf. Chapter III) performance 
data were provided by two or more users. Separate ratios were calculated 
for each user and these were averaged to give a single ratio for the profile 
which was used in calculating the mean value for the population. The 
standard errors, which were computed for all ratios averages, define the 
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statistical confidence of an estimate of service performance, i.e. the 
probability that, within certain error limits, an identical value would be 
obtained if the experiment were repeated with a different, though equally 
representative, user population. 

The Mann-Whitney U Test * ^ was used to test for significant 

differences between two estimates of a performance ratio corresponding to 

different user populations (e.g. different employment categories). Vilcoxon's 
^^^^^^^^^ 1 li 1 ? 

matched-pairs signed ranks test * was used to test for differences between 
two mean values for the same user population (e.g. recall ratios of MAJOR and 
MINOR value items). In all cases, a one-tadled test was used and differences 
were considered significant if the statistical confidence was 9S% or greater 
(a = 0.05)» In some instances values of a are given in parenthesis to indicate 
the degree of significance revealed by the test* The above tests were not 
considered necessary if the difference between two means was greater 
than twice the sum of the two standard errors. This was taken as sufficient 
evidence that the difference was significant at the $S% level . 

S. PRECISION AND NOVELTY RATIOS 

a) Definitions 

number of relevant references retrieved by the service 

Precision Ratio = 

total number of references retrieved by the service 

number of relevant references not already known to the user 

Novelty Ratio = 

total number of relevant refs. retrieved by the service 

Average output sizes (number of references retrieved/issue) were also 
calculated and are always quoted in conjunction with precision figures. The 
output size may be at least as significant an indication of user satisfaction 
as precision. For example, consider two users who are identical with respect 
to all performance parameters except output size. If the profiles have a 
precision of 10% but output sizes of 10 and 100 references respectively, it is 
probable that the former would be considered more satisfactory by the user. 

b) Methods of measuring precision and novelty ratios 

Estimates of precision and novelty ratios are based on relevance decisions 
for all references contained in the computer printout from f o^xr BA Previews 

For v?.iues exhibiting normal distributions, a difference greater than the sum 
of the two standard errors can be t^dcen as significant at the 95% level. 
However, performance ratios for information retrieval systems have non-normal 
distributions. Unless a difference was greater than twice the sum of the two 
standard errors, the mo- 3 rigorous non- parametric tests were therefore used» 
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tapes. These tapes corresponded to two issues of Biological Abstracts (52(15). 

^(16)) and two issues of BioResearch Index (2l(07), 2l(C8)). For each of 

these four issues the participants in the evaluation were supplied with two 

copies of the printout. On one copy they were asked to mark each reference 

as MAJOR VALUE, MINOR VALUE or IRRELEVAl^T and, also, to indicate whether they 

•already know of the existence* of the MAJOR and MINOR VALUE articles. The 

latter information was required for the calculation of novelty ratios. The 

marked copy was then returned to the liaison scientist. The instruction sheet 

regarding this feedback, which was provided for the guidance of users, is 

♦ 

shown in an Appendix . 

Precision and novelty ratios were computed for the Biological Abstracte (BA^ 
issues alone, fo^ the BioResearch Index (BRI) issues alone, and for the 
compTote data base ( ba/bRI ). In computing ratios for the complete data base, 
data from the BA and BRI issues respectively were weighted in a 2:1 ratio 
corresponding to their relative frequency of publication. The collection of 
feedback from two issues each of both BA and BRI was designed to provide 
equally representative samples of the two parts of the data base. The 
procedures for calculating precision and novelty ratios for each profile can 
be illustrated with the figures given in Table IV A for an individual user. 



TABLE IV A. SAMPLE DATA FOR PRECISION AND NOVELTY RATIOS 



1 


BA 


BRI 




52(15) 


52(16) 


71(07) 


71(0B) 


Total no. of rererences retrievrd 




47 


41 


51 


No. of relevant references (MAJOR or 
MINOR VALUE) 


16 


14 


8 


20 


No. of relevant references al:*eady 
known to the user 

i 


2 


5 


4 


5 



3ee Appendix IV. 1 
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For BA only 

Precision (relevant)* = ^ ^j^ = 37.5% 

Output size = 2 ^'^ = ^O'O refs/issue 

Novelty (relevant) = ^''^ " 1I t lil^ " = 33.3% 

For BRI only 

Precision (relevant) = ^ s 30.14% 

etc. 
For BA/BBI 

Preclaic. (relevant) = ^3 t k^] I ImMI ) = ^^.9% 

Output size = 2(^1 ^ h i) * ?1) , 1,2.0 refB/iB«» 

c) Mean Values for precision and novelty ratiosi and for output sizes 

Table IV B gives mean values for precision and novelty ratios, both for 
all relevant references and for MAJOR VALUE references, for these evaluees 
who returned the necessazy data. Avera^ output sizes are also given. 

TABLE IV B MEAN PRECISION ANB NOVELTY RATIOS, AND OUTPUT SIZES 



No. of BA BRI BA/BBI 



tusers'*' 



Output size 


11+2 


3i+.9-2.8 


33.I+-3.2 


31+.1+-2.8 


No. of relevant 
items/issue 


11;2 


11.2^1.1 


10.9-1.2 


ii.iii.o 


% Precision 










Relevant 


11+2 


38.3-2.0 


38.7-2.2 


38.6ii.9 


MAJOR VALUE 


11+2 


12.3^1.2 


10.9-1.2 


12.lil.T 


% Novalty 










Relevant 


^2h 


73.1+^2.1 


81+.8l'1.8 


76.9^1.8 


MAJOR VALUE 


119 


50.8t3.1+ 


71+. 5:^3.2 


57.2i3.o 



Number of evaluees who returned satisfactory feedback. 
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The tezms precision ratio, preci8..on and % precision are treated as synon- 
ymous but all numerical values are given as percentages. The same practice 
is followed for the other performance ratios. 
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It raight be expected that BR I would produce less relevant references than BA 
but Table IV B shows that the average output size and the mean precision of 
LA and BRI respectively are almost identical both for all relevant and for 
Major value references. Predictably, the proportion of article8~not~"previously 
known to the users was significantly higher fo" BRI (84.8^) than for BA 
(73,1,1^) and the difference was even more marked for I^IAJOR VALUE references 
(7r. 5^ for BRI; 50.8^ for BA). 

One third of all relevant references were judged to be of MAJOR VALUE 
and the novelty ratio for MAJOR VALUE items (57.2^) was significantly lower 
than for the relevant (MAJOR + MINOR VALUE) items (76.9^). 

d) Variables influencing the novelty ratio 

Values for the novelty ratio, which defines the proportion of references 
found by the service which were not previously kno'.vn to the user, might appear 
to be an excellent indicator of user satisfaction, but caution must be 
exercised if this parameter is to be interpreted in these terms. The novelty 
ratio of a current awareness service is a composite measure, the value of 
which is determined by the effectiveness of the service, relative to the 
user's other current awareness methods, with respect to currency, recall and 
coverage. 

Thus, a fast alerting service such as Chemical Titles or ASCA ( Automatic 
Sub.iect Citation Alert ) which may report references before the user sees them 
in the primary literature, is likely to exhibit a high novelty ratio, 
irrespective of the recall and coverage performance of the service. On the 
other hand, the novelty ratio will be determined mainly by recall and coverage 
performance if the S.D.I, service is less current than a user's 'other methods*, 
as is more likely to be the case for data bases in which there is substantial 
intellectual input to the indexing and/or abstracting processes. In such 
cases a high novelty ratio could be taken as an indication either of the 
retrieval effectiveness oV the S.D.I. service, or of the inadequacy of the 
users' other current awareness methods. 

Though no measures of relative currency were made, it is reasonable to 
asr.urae t,h-.t the BA Previews service is substantially less current than the 
user's own current awareness methods. Evidence in support of this assumption 
io provided by the samples of 'references found by the user's existing current 
awureness methods' which were obtained for the purpose of computing recall and 
coverage ratioy. This evidence is discussed in a later section of this chapter 
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(page 62). It seer.s probable, therefore, that the high value (77;^) of the 
novelty ratio for the BA Preview, service is detPrr^ined more by its relative 
recall and coverage performance than by its relative currency. 

It is tentatively suggecced that, if the effect of currency can be 
ignored, the novelty ratio of a service such as BA Previews can be construed 
as a measure of the effectiveness of the user's conventional current nwareness 
methods. In the present experiment, this would indicate that the user id 
retrieving only 2% of relevant literature in his subject area by conventional 
methods. This may in part be due to the fact that users were less 
conscientious in their manual searching as a rer.uU of receiving a BA Previews 
service. """""" 

Replies to the final questionnaire (of. Chapter V) showed that 92^ of 
users recognised that the service retrieved references which they would not 
Otherwise have found. 

^) Additional estimates of precision 

The precision ratios given in Table IVB are based on precision samples 
in Which the evaluees identified the individual MAJOR or MINOR VALUE references 
appearing in their outputs. Additional precision data, for both evaluees 
and non-evaluees. can be obtained from the feedback cards (Chapter III, mge 
36) on which the user specified only the number of MAJOR or MINOR value items. 
Table IVC shows precision ratios for the evaluees, computed from data provided 
on the feedback cards during two one-month periods (r.onth A .nd month B) which 
irmnediately preceded the collection of the precision sample. Each one-month 
period corresponds to two issues of BA and one of BRI. Also given in the 
table are the precision ratios computed from the combined dnta for both 
months, for both evaluees and non-evoluees . 

TAi-LP; TV ADDITIONAL PRFJCJiUOM R;Vri03 FOR 
KVALTclEo A in; NON-rJVALUKKG 
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The following conclusions were drawn from these additional precision 
ratios: 

(i) There is no significant difference between the estimates for 
month A (40.2%) and month B (38.4%). A one month sample is sufficient 
to provide a valid estimate of precision. 

(ii) Estimates (for A and B combined) for evaluees (39.0%) and non- 
evaluees (40.2%) are almost identical. In relation to precision 
performance, the evaluees are, therefore, an effectively random sample of 
the full population. 

(iii) For evaluees, the precision ratio provided by the feedback cards 
(39.0%) does not differ significantly from that obtained from the 
precision sample (38.6%). Hence, the precision estimates are not 
influenced by the method of collecting the precision data, i.e. whether 
a user is asked to give only the number of relevant items, or to 
identify each reference. 

6. RELEVANCE PREDICTABILITY 

Relevance predic;;ability is a measure of the extent to which a document 
surrogate (e.g. a title or an abstract) allows the user to make a correct 
prediction of the relevance of the actual document. It is expressed in terms 
of a series of conditional probabilities^ ' \ For example: 

-> D^) = the probability that, given the user has judged a 
title (t) to be relevant (r), he will judge the 
document (d) to be irrelevant (x). 

The measurement of these probabilities for BA Previews required two 
relevance judgements on the same item. The user had first to decide the 
relevance of an article on the basis of the information provided in the 
BA Previews printout and, subsequently, he had to provide a judgement based 
on the full document. These dual judgements were obtained for samples of 
7-12 articles for each user. The samples were a random selection of the 
English -language articles from serial publications which appeared in the 
computer printouts for the BA issues ^(l5) and ^(l6). These printouts 
formed part of the precision sample (cf. page Sk) and evaluees had already 
provided relevance judgements based on the enriched title which appeared in 
the printout. The corresponding documents, which were obtained from the 
National Lending Library, were subsequently distributed to all evaluees. 
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Relevance judgements on 1049 documents were returned by 133 evaluees . The 
average interval between judgements on the surrogate and on the actual 
document was 2 months. 

It must be emphasised that relevance predictability data were obtained 
from reference samples which were not fully representative of the material 
covered by BA Previews . The samples were restricted to English-language 
articles from serial publications, and items from BioResearch Index were 
completely excluded. 

Table IV D gives relevance predictabilities calculated from the user 
relevance decisions. The subscripts A, B and X are used to denote MAJOR 
VALUE, MINOR VALUE and IRRELEVANT items respectively. " 

TABLE IV D. RELEVANCE PREDICTABILITIES 

(three relevance categories r 



Sample Size 


P (T, 


— > 


^A^ 


0.816 


135 


P (T, 


-> 


^B^ 


0.169 


»MAJOR VALUE* titles 


P (T, 


— > 




0.015 




P (Tg 


— > 


^A^ 


0.185 


294 


P (Tj 


— > 


^B> 


0.682 


»MINOR VALUE* titles 


P (Tg 


— > 




0.133 




P(Tjj 


— > 


^A^ 


0.027 


620 




— > 


^B^ 


0.198 


'IRRELEVANT* titles 


P (T, 






0.775 





All mean values are number avera, ^ 



The instruction sheet accompanying the documents is shown in Appendix IV 2. 
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It will be seen from this table that transitions between the MAJOR VALUE 
and the IREELEVANT categories are infrequent: only 1.5% (2 items) of MAJOR 
VALUE titles were subsequently Judged to be irrelevant documents, and only 
2.7% (17 items) of IREELEVANT titles were judged as MAJOR VALUE documents. 
Transitions between MINOR VALUE items and the other categories were more 
common. However, a clearer overall picture is provided by relevance predict- 
abilities based on a bipartite classification (relevant or irrelevant) as 
shown in Table IV E. Both number and ratio averages are given in this table 
and it can be seen that the differences between the two are small. 



TABLE IV E. RELEVANCE PREDICTABILITIES 
(TWO RELEVANCE CATEGORIES) 







Ratio 
average 


Number 
average 




V 


0.899 


0.901, 


P (Tj, -, 




0.101 


0.096 


P (T^ -:. 


V 


0.211 


0.225 


P (T^ -> 




0.789 


0.775 


P (T -> 


B ) 




0.828* 



the probability of the two decisions 
being the same 
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It is evident that the probability that an irrelevant title will correspond 
to a relevant docxment (0»211) is significantly greater than the probability 
that a relevant title will correspond to an irrelevant doc\ament (0»101)» 
Furthermore, there are rather more irrelevant titles than relevant ones in a 
user's output. It appears, therefore, that there is a greater likelihood that 
the user will miss relevant material because the information provided by the 
title is insufficient, than that he will waste time in looking up documents 
which turn out not to be relevant* 

The figure in the last row of Table IV E (0.828) defines the probability 
that the user will make a • correct* relevance decision based on the title, i.e* 
it will be the same as his judgement on the full document. 

Table IV P shows the two precision ratios for the document sample, 
computed on the basis of title and document relevance decisions respectively. 
The precision ratio provided by the fixll precision sample (cf. page Sk) is 
also given. 



- 60 - 



TABLE IV F. PRECISION RATIOS BASED ON TITLE AND DOCUMENT 
RELEVANCE DECISIONS 



Sample 


Relevance Decision 


% Precision 


Document sample 


FULL DOCUMENT 


49.1-?. 2 




TITLE 


40.6-2.4 


Precision sample 


TITLE 


38.6-1.9 



Two points of interest emerge: 

(i) Precision ratios based on title decisions for the document and 
precision samples respectively are not si.^ficantly different, in 
spite of the fact that the document sample is limited to English- 
language articles from serial publications. 

(ii) The precision ratio based on documerl; decisions (49.1^) is 
significantly higher than that for title decisions (40. 6^^). 
However, the improved performance suggested by the former ratio is 
probably illusoiy from the user's viewpoint. If he is unable to 
recognise that an article is relevant, he is unlikely to make practical 
use of it. 

7. RECAU AND COVERAGE 
a) Definitions 

Recall ratio = ^^^m^e^? of relevant refs. re*,rieved by the service 

number of relevant refs. in the data base 

number of relevant refs. in the data base 



Relative coverage 
ratio 



total number of relevant refs. in source X 
where X - a suitable alternative source 

The coverage ratio is a relative -neasure, the value of which will be 
dependent on the source from which the reference sample was obtained. In 
contrast, the recall ratio, as defined -ibove, is a characteristic of the 
service alone and is independent of the source of the recall sa-rple. However, 
a valid estimate of this measure will be obtained only if the rec-a 1 aa-n.pie 
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is equivalent to a random sample of the relevant references in the data 
base. For example, a search of the printed indexes of BA/BRI would give a non- 
random ('biased') sample. The manual search would probably use the same index 
entries which appear in the search profile, and would therefore be likely to 
retrieve the same references as the profile. A spuriously high recall ratio 
would result. In almost all circumstances a biased recall sample will give a 
recall estimate which is significantly higher (rather than lower) than the 
true value. As will be shown (page 70),at least some of the recall samples 
obtained in the present experiment were biased. 

b) The reference samples 

In the BA Previews evaluation, each evaluce was asked to provide two samples 
of relevant references. Each sample provided estimates of both recall and 
coverage ratios. The two samples were: 

(i) Relevant references found by the user in the course of his normal 
current awareness searching. These references were listed on a form 
(bV-6) , which was supplied by the liaison scientist, and they will be 
referred to as the BV-6 sample . 

(ii) Relevant references retrieved by an 'alternative service*. 

c) The BV-6 sample (references found by the user) 

Each evaluee was asked to supply a list of up to 30 relevant (MAJOR or 
MINOR value) references found in the course of his normal searching of the 
current literature. The instruction sheet and feedback forms sent to users 
ive shown in an Appendix . Users were also asked to indicate the source from 
which the reference had been obtained (e.g. primary journal, specified abstract 
.iournal). The different sources mentioned by the users and the proportion 
of references contributed by each source are shown in Table IV G. 



The sample need be random only with respect to the probability of a 
reference being retrieved by the service. 

Appendix IV. 3 
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TABLE IV G. SOURCES IN WHICH USERS POUND REFERENCES 
FOR THE BV-6 SAMPLE 



Source Proportion of references 



Primaiy journals 


55^ 


Current Contents 




Abstract journals, reviews 


11% 


Internal information service 
(e.g. library bulletin) 


Offo 


Personal communication 




Other 


2% 



d) Evidence regarding the relative currency of BA Previews provided 
by the BV-6 samples 

Our conclusions (page 55) regarding the estimated novelty ratio for 
BA Previews are dependent on the assumption that BA Previews is, on average, 
less current than the user's own methods of searching the current literature. 
The predominance of primary journals and of Current Contents as sources of 
references found by the user (Table IV g) lends support to this assumption. 

More direct evidence is provided by the dates of appearance in the 
printed issues of BA and BRI of those articles in the BV-6 sample which were 
found to be covered by the data base. Table IV H summarises the appearance 
dates for 1^93 articles which were reported on the BV-6 forms returned by a 
subset of 1^2 users. The number of articles appearing in individual months 
are given in an Appendix . 



Appendix IV. 6 
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TABLE IV H. DATES OP APPEARANCE IN BA PREVIEWS OP 
REFERENCES PROM THE BV-6 SAMPLES 



Month of Appearance 


No* of references 


% of references 


April 1971 or earlier 


us 


9% 


May - Aug 1971 


121 


25% 


Sept - Dec 1971 
Jan 1972 or later 


288 
39 


1 66% 


Total 


1)93 


100% 


Based on the printed publication dates given on the printed 
issues of BA and BR1« The corresponding BA Previews tapes 
usually arrived in the UK during the same month, but there 
was a delay of 2 - 6 weeks before the user received the 
search output ♦ 



Most of the BV-6 forms were returned in August 1971 and contained articles 
found by the user in June, July or Aiagust* The data given in Table IV H 
therefore indicate that, for at least 66% of the BV-6 references, BA Previews 
is less current than the user^s own current awareness methods^ For only 
9% of the references is it reasonably certain that the computer search would 
find the reference first* 

e) The alternative service samples 

All evaluees were provided with an output from one of three altemative 
services and were asked to mark all MAJOR or MINOR VALUE references on the 
output, which was then returned to the liaison scientist* The instruction 
sheets relating to each of the services, which were issued to participants, 
are shown in an Appendix"*"* (it mxist be emphasised that the alternative 
services were being used only to provide recall and coverage samples for the 
BA Previews service* No attempt was made to optimise search strategies for 
the altemative services ^ and our experimental data do not allow comparisons 
to be drawn between BA Previews and any of these services)* The three 
services were: 

(i) A corrrputer search of a 3-month (April - June, 1971) MEDLARS file* 
This was supplied to ^1 users with biomedical interests for whom MEDLARS 
( index Medicus) was considered an appropriate altemative data base* 



^ Appendix IV*): 
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(ii) A computer search of 4 weekly issues of the organic and biochemistry 
sections of Chemical Abstracts Condensates (^), which was supplied 

to 50 users with biochemical interests. The four issues were those 
published in February and March 1971 (Vol. 74; issues 5,7,9,ll). 

(iii) A manual search of Science Citation Index (SCl ) for the period 
January - March 1971. Output from this search was supplied to the 
remaining 55 participants. Recipients of this service had previously 
been asked to provide a list of up to 15 important references or 
authors for use as index entries in the :.earch of the citation index. 

Search profiles for the MEDURS and CAC services were written by the 
liaison scientists. The SCI search was undertaken by the Institute for 
Scientific Information (iSl) in Philadelphia. 

^) Analysis of reference sampi ea 

Reference samples supplied by 80 of the evaluees were analysed. The 
two reference samples (the BV-6 and the alternative service samples) provided 
by each of these users were analysed separately. Two recall ratios and two 
coverage ratios were therefore obtained for each of the 80 users. 

Articles from non-serial sources (books, patents, U.S. government 
reports, conference proceedings, etc) were excluded from both coverage and 
recall samples. Estimates of both recall and coverage samples were therefore 
based on relevant references from serial publications only. 

Table IV I gives the total number of references (for 80 users) in the 
different samples which were used in the analysis. The complete sample 
referred to in Table IV I (row [2]) is the total number of references reported 
by the users, not all of which were actually analysed. 

The instructions given to users (cf. page 61) ensured that no BV-6 
s.-nple contained more than 30 references. Furthermore none of the SCI senrches 
yielded more than 30 relevant references. However, the CAC ond MRDLARS 
profiles often produced many more than 50 relevant items. For the piorpoaes 
of our analysis, the number of items in the CAC und MKPLARo -amples was 
restricted to not more than 35 references by oelecting a subset of the 
complete sample whenever this was necessary. The ter.-r, experinental sample 
(Table IV I, [3j) was adopted for the referen-es which were selected for 
-maly.is. In the case of the BV-r. and JCI .ampler., the complete s.r.ple and 
the experimental simple are identical. 
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TABLE IV I. RECALL AND COVERAGE RATIOS - SAMPLE SIZ''"3 





SOURCE OF SAMPLE 




BV-6 


SCI 


CA 


MEDURS 


ALL SERVICES 


[l] No* of users 


80 


27 


27 


26 


80 


[2] ComDlete samDle 


1346 


346 


868 


151.8 


4108 


[3I Experimental sample 


1346 


346 


591 


668 


2951 


[4] Mean sample size 
(ref s/user ^ 


16.8 


12.8 


21.9 


25.7 


36.9 


[5] No. of non-serial items 


63 


0 


127 


0 


190 


fel Covera*re sample 


1283 


346 


464 


668 


2761 


[7] No. of items covered 


950 


292 


280 


482 


2004 


fsl Recall sample 


705 


266 


184 


345 


1500 


[9] No. of items retrieved ^ 


414 


119 


151 


190 


874 



The Coverage Sample; for each user, the coverage sample consisted of a„l 
items in the experimental sample with the exception of non-serial iteps. 
Each article in the alternative service samples and in approximately one-half 
of the BV-6 samples was checked against the printed BA and BRI for the period 
January 1970 to September 1972 inclusive. Articles from the remaining BV-6 
samples were checked for the shorter period January 1970 to February 1972 
inclusive. The procedure f^r determining whether or not an article is covered 
by BA Previews is detailed in an Appendix*. 

A list of the source journals scanned by BIOSIS is published annually"^. 
However, not all articles from a journal included in the source list are 
necessarily selected for inclusion in the data base and, furthermore, articles 
from a single issue of a primary journal may be distributed over several 
issues of ^ Previews. (Some data on the currency scatter of articles from 
the same issue of a primary journal is presented in an Appendix"^). 



* 
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If an article is not found in the data base during a given period, 
there is, therefore, no exact criterion for deciding whether or not the 
article will appear in some later issue of the data base, outside the period 
during which the data base was searched (we can be certain that an article 
has not appeared in the data base prior to the period searched, if we search 
back as far as the publication date of the article, if this is known). 
However, a rough and ready statistical criterion is provided by the distri- 
bution of the appearance dates in BA Previews of those articles which were 
located in the data base. The distributions of appearance dates for the 

BV-6 samples and for each of the alternative service samples are given in 
♦ 

an Appendix . These data provide strong evidence that the proportion of 
items appearing outside the period searched is too small to exert any signifi- 
cant influence on the coverage estimates. For example, the 482 'covered* 
references in the MEDLARS sample were distributed as fellows: 

1970 1971 1972 

Jan-June July-Dec Jan-June July-Dec Jan-Ju^:- July-Sept 

1.9^ l.Ofo 50.2^ 59.6^ 6.2 l.O^ 

Articles in the BV-6, CAC and SCI samples showed an even smaller proportion 
of references in the last quarter (July-Sept 1972) of the search period. 

The method for computing a coverage ratio for each user, from either of 
his two samples, can be illustrated as follows: if there are 20 references 
in the coverage sample and 15 are located in BA Previews then the coverage 
ratio is 15/20 = 75?^. 

The Recall Samples : theso are subsets of the coverage samples and consist 
of references known to be covered by the data base, with the following 
exceptions: 

(i) Articles were excluded from the recall sample if they appeared in 
the BA Previews data base prior to the last profile amendment and hence 
before the profile was operating at its 'optimum* level. 

(ii) Articles in the BV-6 sample (but not the alternative service 
samples) were excluded if they appeared in an issue of BA Pr eviews 
from which the user had received a computer printout before he returned 
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the BV-6 form. Tliis ensured that the rec€dl sample did not include any 
references which the user had actually found in his search output. 

As shown in Table IV I, out of the 200 references known to be covered 
by the data base, 501, {2S%) were excluded from the recall sample for one or 
the other of the above reasons. 

The BA Previews issue in which each article in the recall sample 
appeared had already been identified during the processing of the coverage 
sample. Each of these articles was checked against the user's computer 
printout for that issue in order to determine whether or not the article had 
been retrieved, and a recadl ratio was then calculated. For example, if the 
recall sample contains 16 references of which 12 appeared in the user's 
printout, the recall ratio is 12/l6 = 75%. 

g) Estimates of Coverage Ratios 



TABLE IV J. COVERAGE PERFORMANCE OF BA PREVIEWS 





(80 users) 


MEDLAHS 
(26 users) 


SCI 
(27 users) 


CAC 
(27 users) 


% non-serial items 










[l] alternative service 
sample 




0.0 


0.0 


27.8 


[2] BV-6 sample 


3.8 








Coverage ratio (serial 
items) 










i_3] alternative service 




72.3^2.8 


83.5^14.8 


59.2^3.3 


[I] BV-6 


75.14-2.1 


77.1-2.8 


72.2iu.6 


76.6^3.2 


% coverage failures - % 
from journals not in 
source list 










[5] alternative service 




12.6 


17.1 


38.9 


[6] BV-6 


15.7 


13.a 


IS. 5 





The first two rows of Table IV J give the percentage of articles in the 
experimental samples which were classified as ♦non-serials* and were therefore 
excluded from both coverage and recall samples. 

Coverage ratios for serial items are given in rows [3j suid [^-l of the 
table. Mean values are given for each of the three subsets of evaluees 
corresponding to the three alternative services. For the BV-6 sample, a 
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coverage ra.Uo for the whole population (80 userc) is also given. For each 
subset, coverage ratios were computed both for the alternative service 
samples and for the BV-6 samples. For example, the column headed ''MEDLAfiS* 
gives mean valueo for the 2^ users who received a MEDLARS profile. The 
figure in row [3] is the % coveraf^e relative to MEDLAHS (72.39{); the figure 
in row [i;] is the % coverage relative to the user»s own methods of current 
awareness searching. 

Relevant items not covered by BA Previews ('coverage failiires*) were 
classified into two groups: 

(i) articles from Journals not listed in the BA Previews soixrce lists 
(1970-1972)* and hence not scanned by BIOSIS. 

(ii) articles which appeared in Journals scanned by BIOSIS but which were 
not selected for inclusion in the data base. 

Rows [5] and [6] of Table IV J show the proportion of coverage failures which 
were in the former category. 

The coverage ratios computed from the BV-6 samples are approximately 
the same for all three subsets of users. Tne BA Previews date base covers 
about 1$% of the references found by the user in the course of bis normal 
current awareness searching and, in this rtispect, users with biomedical or 
biochemical interests do not differ significantly from core biologists. 

The CAC sample gave a lower coverage ratio (59.2%) than xhe MEDLARS 
sample (72.3%). This difference is almost entirely accounted for by the 
number of articles in the CAC sample which were from journals not scanned 
by BIOSIS. These accounted for 39% of CAC coverage failures as compared to 
13% of the MEDLARS and 16% of the BV-6 (all users) coverage failures. It 
appears that the Journal coverage of ^A Previews does not extend as far into 
biochemistry as it does into biomedicine. 

However, caution is always necessaiy in interpreting relative coverage 
ratios. The coverage ratio of BA Previews relative to CAC is a meaningful 
measure only if CAC is a reasonable choice of an alternative service for the 
particular user population (it is assumed that BA Previews is a reasonable 
choice by virtue of the user's participation in the experiment). Thus, the 
relatively low value for the coverage ratio would have little significance 
if CAC was retrieving only 1 relevant item/month compared to the 36 relevant 

* Excluding the •archival* listings (L09 titles in 1971); these are titles 
which were covered iJi the past but are now considered unsuitable or are 
unavailable. Archival titles were classified as 'not covered'. 
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items/mon-'w*! retrieved by BA Previewa « In fact, from the complete sample size 
given in Table IV I, we can compute that CAC is retrieving about 16 articles 
a month. (This makes no allowance for the fact that we did not attenq?t to 
optimise the CJAC profile.) 

It must also be noted that we are not in a position to compare the 
coverage of BA Previe ws with that of CAC (or of SCI or MELLAES). Comparisons 
could be made only if we had also measured the coverage of CAv relati^^e to 
BA Previews. However, on the basis of the figures given in the previous 
paragraph it seems possible that many users with biochemiceil interests will 
require access to both data "bases. 

Only a minority of coveragb failures were from journals which are not 
scanned by BIOSIS. For example, only l5.7% of coverage failures from the 
BV-6 sample were of this type. The majority of coverage failures were 
articles which had beon consciously excluded from the data base either on 
the grounds of quality or because they were judged to be outside the subject 
area of the data base. 

The journal distribution of the coverage samples was also investigated. 
The ■ Oil articles in the coverage samples which were known to be covered by 
BA Previews were distributed among 61S different primsuis' journals. The 
coverage sample also contained 1^1 articles from 86 different journals which 
do not appear on the BIOSIS soxirce list. A fiall list of these 86 joimials 
appears in an Appendix*. Three of these Journals provided relevant references 
to three different users, and four provided references to two users. However, 
the remaining 79 joumalp contributed relevant material to only one user in 
eacl.' case. Thtr© appears to be no case, therefore, for extending the Journal 
coverage of BA Previews . 



* Appendix IV. 8 



- 70 - 

h) Estimates of Becall Hatioa 



TABLE IV K % RECALL, % PRECISION ANB OUTPUT SIZE FOR 80 USERS 





MEDLARS 
(26 users) 


SCI 
(27 users) 


CAC 
(27 users) 


(80 lusere) 


9^ Recall 










Alternative service 


59.5^14.7 




82.7-2.9 




Kefs found by the user 
f BV-6 f annile'i 




63.5^5.9 


69.1^14.5 


62.5^3.2 


% Precision 


iil4.9i5.0 


39.lil4.l4 


37.5^3.1 


l40.li2.1+ 
(38.6il.9)* 


Output Size 


39.3^5.8 


::>.oi5.5 


30.1^5.7 


32.9^3.3 
(314.14^2.8)* 


a) Mean values for II42 users, cf Table IV B 



Table IV K shows the recall estimates for the two recall samples supplied 
by each user. Also given are the % precision and output size for each 
subset of u.3ers. The mean precision and the mean output size for the 80 
users for whom recall ratios were oonqputed do not differ significantly from 
the mean values for the II42 users who supplied precision figures. 



We have already pointed out (p. 60 ) that a valid estimate for the 
recall, ratio, as we have defined it, can be obtaineu only if the recall 
sample is 'unbiased* and the estimate is therefore independent of the source 
of the sample. If this condition is fulfilled then, for a given subset of 
the population, the BV-6 and the alternative service samples should give 
values which are identical within the statistical confidence limits of the 
two estimates. 

The recall ratio from the CAC sample (82.7%) is significantly higher 
than that from the BV-6 sample (69.1%) for the same subset of users. 

We selected CAC as an alternative service because we wished to obtain 
a relative coverage estimate for users with biochemical interests and it 
had been expected that this sample would give a spuriously high value for the 
recall ratio. Articles are retripvpd from CAC by searches of author titles 
and of keywords added by indexers. CAC profiles therefore tend to retrieve 
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the same references as those found "by searching the enriched titles of 
^ Previews , and a biased recall sample results. 

It was also observed that the recall estimate from the SCI sample 
{hh*0%) is significantly lower than that from the corresponding BV-6 sample 
(63.5%). It has been argued elsewhere by Leggate^^^ that a recall sample 
can normally only be ♦biased' in one direction and this bias will always 
result in a spuriously high value for the recall ratio. If this view is 
accepted then we must conclude that the BV-6 sample is biased and gives 
recall estimates that are hi^er than the true values. Nevertheless, the 
remarkably- low estimate provided by the SCI sainple gives rise to some 
suspicion as to whether biased sampling may, in some circumstances, result 
in too low a value for the recall ratio. 

If we accept the BV-6 recall ratios as a reasonable basis for comparison 
between subsets of the population, then it is of interest that profiles for 
Causers exhibit higjier recall (69.1%) but lower precision (37.5%) than the 
profiles for MELLABS users (recall: Sk.h%l precision: kh*9%)' 



i) Hecall an d Coverage for MAJOR and MINOR value references 

TABLE IV L RECALL AND COVERAGE RATIOS FOR MAJOR AND MINOR 
VALUE ITiMS (BV-6 SAMPLES, 80 USERS) 





% Recall 


% Coverage 


MAJOR value 
MINOR value 


66.1^^3.8 


76.8^2.5 
714.6^2.7 



The figures given in Table IV L illustrate the variations in coverage and 
recall performance associated with different degrees of relevance. For items 
in the BV-6 samples the % recall is significantly higher for MAJOR value 
iSS,h%) than for MINOR value items (55. 9%) » but the coverage ratios for the 
two categories are effectively the same. A similar picture was presented by 
the alternative service sauries. 

8. OVERALL RECALL 

Relative overall recall s 

number of relevant references retrieved by the service 
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= recedl x relative coverage 

The recall and coverage ratios correspond to two quite distinct aspects of 
service perfoimance and the values of each are determined by different sets 
of variables. Nevertheless, the overall recall ratio, which is the product 
of recall and coverage ratios, is probably the more meaningful measure from 
the user's point of view. The user is presumably interested in the 
proportion of relevant material which is 'missed', irrespective of whetter 
a missed reference represents a recall or a coverage 'failure'. 

From the recall and coverage ratios given by the BV-6 samples we 
computed a value of k7% for the overall recall ratio relative to the user's 
noimal methods. The BA Previews service is retrieving less than half of the 
relevant literature found by the user' s normal searching methods. On the 
other hand the novelty ratio (779^) indicates that the user was finding only 2}% 
of the relevant material retrieved by the S.D.I. service. It appears that the 
BA Previews service and the user's own search methods should be regarded as 
complementary rather than competitive. It is, however, unlikely that the 
retrieval of all relevant material is necessary for complete user satisfaction, 
except in the case of information centres and particular types of research 
workers, e.g. taxonomists. 

9. RETRIEVAL P EBFORMANCE OF PROFILES WRITTEN BY DIFFERENT LIAISON SCIENTISTS 

Table IV M lists the key performance measures for the subsets of profiles 
written by each liaison scientist. 

TABLE IV M PERFORMANCE MEASURES FOR PROFILES WRITTEN 
BY THE THREE LIAISON SCIENTISTS 





Liaison Scientist 




JS 


BS 


MW 


Output size 
(refs/issue) 


35.9^7.2 (37)* 


32.1^3.2 (62) 


37.3^3.5 {U3) 


Y: Precision 
(relevant) 


3U,}'k>^ (37) 


l4l.0i2.9 (62) 


39.8i3.5 {h3) 


% Recall (relevant, 
BV-6 sample) 


59.3-6.8 (20) 


62.2i5.5 (30) 


65.lii4.7 (28) 


Novelty (relevant) 


79.7^2.7 (26) 


73.6i2.l (53) 


75.8i3.1 {h2) 


* Figures in parentheses indicate the number of profiles which 
contributed data to each mean value. 

— — — — j 
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The only statistically significant difference among the liaison 
scientists was in the precision figures for JS and BS (a = O.OU). Even 
this difference is numerically smcdl and we conclude that there are no 
dramatic differences in the performance of profiles written by different 
liaison scientists, even though there are substantial differences in the 
number of terms and the complexity of the search logic (cf p. 21 ). 

10. PERFORMANCE OF PROFILES WRITTEN FOR USERS IN DIFFERENT EMPLOYMENT 
CATEGORIES 

TABLE IV N PERFORMANCE MEASURES PGR USERS IN THREE EMPLOYMENT CATEGORIES 





Employment Category 


UNIVERSITY 


GOVERNMENT • 


INDUSTRY 


Output size (refs/user) 

% Precision (relevant) 

% Recall (relevant, BV-6 
sample) 

% Relative Coverage 
(re levant, BV-6 sample) 

% Novelty (relevant) 


39.8ii4.5 (55) 
35.6i2.8 (55) 

55.2ii4.9 (3i4) 

80.9-2.7 (3i+) 
79.2^2.9 (i+6) 


28.7^3.6 (62) 
141.7^2.9 (62) 

67.6i5.8 (28) 

71.lii4.0 (29) 
76.9^2.3 (53) 


1+1.0^12.0 (18) 
ill. 6^7.2 (18) 

73.8^7.9 (10) 

68.9^5.5 (10) 
75.9-ii.3 (16) 



Statistical tests were made for significant differences between mean 
values of the performance parameters for university and government users 
respectively. We found that the BA Previews services supplied to the 
university population had larger outputs (a = 0.02), lower precision (a = 0«06), 
lower recall (a = 0.05), and higher coverage (a = 0.03)- It will be noted that 
the difference in precision values is not significant at the 95% level. In the 
case of the other parameters the magnitude of the differences is not large and, 
furthermore, we can offer no hypothesis to explain these differences. We 
decided, therefore, that there is insufficient evidence to support the 
conclusion that government and university research workers represent 
substantially different populations with respect to S.D.I, services. 

A comparison of university and industrial users showed that the latter* s 
profiles exhibited significantly higher recall (a - O.O3) and coverag:e 
(a - 0.02). However, the mean values for recall and % coverage of 
industrial users are derived from data for 10 users only and we considered 
this to be too small a population for any definite conclusions to be drawn. 
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CHAPTER V 
USER REACTION TO THE SERVICE 

1. RESEARCH AIMS 

The basic piirpose of a service such as BA Previews is to satisfy the 
users' information needs. An essential part of the evaluation was, therefore, 
the collection of information about users' attitudes to the service. 
For this purpose a questionnaire* was designed which would try to answer 
the following questions! 

(a) What did the user consider to be the medn advantages and 
disadvantages of the service? 

(b) V/hat impact did it have on his existing literature searching 
practice and, in particular, his usage of the printed 
Biological Abstracts? 

(c) Would he be interested in subscribing to a permanent current 
awareness service? 

(d) Would he be interested in a retrospective search service? 

2. RESEARCH METHOD 

A pilot study was carried out in August 1971 in order to test a 
preliminary questionnaire. The latter was sent to 15 selected users v^o were 
subsequently interviewed by the liaison scientists. Comments were also 
invited from the members of the Advisory Committee and a few librarians 
with experience of user studies. 

The final questionnaire was distributed to users in September 1971. 
A modified version was sent to the eight information centres (See Chapter Vl). 
Users were asked to complete the questionnaire prior to the final interview 
so that their replies could then be discussed with the liaison scientist. 
This procedure was designed to prevent the liaison scientist influencing 
the user's replies, whilst at the same time the final interview provided 
an opportunity to discuss any difficulties experienced by the user in 



♦ 

See Appendix V.1 
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completing the questionnaire and to clarify ambiguous or apparently- 
inconsistent replies. Written instructions were provided for the guidance 
of the liaison scientist in these discussions. The minority (lO%) of 
users for whom it was not possible to aurrange interviews were asked to 
return the completed questionnaire by post. 

Completed questionnaires were obtained from 280 users. In addition, 
seven of the ei^t information centres returned their modified questionnaire. 

A computer program was written in order to facilitate the analysis 
of the questionnaire replies. We were therefore able to undertake a more 
detailed analysis than would have been feasible by manual methods alone. 
The program provided counts of individual replies and also of combinations 
of replies to different questions. It also tested for correlations between 
different replies by means of the chi-squared test/^^^and for correlations 
between questionnaire replies and values for performance parameters by 
computation of Wilcoxon's U-statistic Unless otherwise indicated 

a correlation is described as significant if the significance level (a) 
is 0.05 or less 9$% confidence that the correlation is not a random 
event ) . 

3. ADVANTAGES AND DISADVANTAGES OF THE SERVICE (Question I) 

In order to find out what the users considered to be the main 
advantages and disadvantages of the BA Previews service, the questionnaire 
listed seven potjsible advantages and nine possible disadvantages. Users 
we2?e instructed to; 

tick only those which you consider to be really IMPORTANT 
to you. Do not tick any which are only of minor significance. 
You may tick none, one or several in each list" (Q. I) 

Their answers are shown in Tables V A and B. 



Two-tailed '-.est. 
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BEST COPY mm\i 



TABLE V A. ADVAMTAGKS OF THE BA PREVIE^.VJ SERVICE* 



1 


Huvanuage ox tne service 


No. of users 
ticking this 




1 (a) 


The profile retrieves references of major value 






1 
j 


which I might not otherwise locate 


135 


48 


1 M 

J 


The profile retrieves background references 






t 


which I might not otherwise locate 


198 


71 


; (c) 


The service covers journals which I would not 








normally see 


219 


78 


•■ (d) 


The Service retripvp^^ T»proT»onr>oc! ^^^r^rr* 








journal publications. e.fr. books, svmnnsirq 




37 


j 


conference proceedings, etc . 






i (e) 


Th© service is timp— savi my i o t -t- T^^^Hn^-^^^o -♦-Vi^ 
*** w^x»j.v^^ ''•xmc—oci V X xig, X • t? « xu X eu-uceb tne 






1 


time spent searching the literature by normal 


109 


39 


i 


methods 






(f) 


The service id convenient, i .e . it obviates the 








need for copying out references from abstracting- 
and indexing journals for incorporation in -riy 


44 


16 




persoa-il inaex 






(e) 


The provicion of a regular service gives r..e 








confidence; rr.y literature searching i^:; no longer 


88 


31 




dependent upon how and when I can do it ^y:;;eir 







All t'r. ei.--r;t u.xt..- .'.lentior.e'i one or -.iore viv.r: Vi^.'-es 
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BEST COPY AVAILABLE 





TABLE V B. DISADVANTAGES OF THE BA PREVIEWS 


SERVICE* 






Disadvantage of the service 


No* of users 
ticking this 




(o) 


The profile retrieves too many irrelevant 


l-O 


45 


(k) 


The profile misses too many relevant references 


27 


10 


(1) 


There is too much delay between publication of an 
article and its aDDearance in "RA Prpvipwcs 


13.? 


47 


(m) 


The journal coverage is inadequate 


26 


9 


(n) 


Many relevant references are of little or no 
practical value: 








(i) they are from journals which are not 
easily accessible 


48 


17 




(ii) they are from publications which I 

v^U Xiox Uc;X UU D6 OI 10 W quail oy 


24 


9 




(iii) they are from foreign language journals 
which I cannot read or afford to have 
translated 


60 


21 


(o) 


The computer printout is unattractive 


78 


28 


(p) 


The computer printout is difficult to file 


110 


39 


(q) 


The coding of the profile is difficult to under- 
stand so I am not sure how the profile logic is 








intended to work>i»e, which single search term 




1 1 
1 1 




or combination of search terms will retrieve o. 








reference 






(r) 


There is not enough information on the printout: 








(i) no abstracts 


T> 


27 




(ii) no author addresses 


76 


?7 




(iii) no indication of the language 
of the original paper 


105 


57 



All but nine users mentioned one or :r.ore xisaavantages 



Advantages : In addition to those listed, three users stated other 
advantages • 

It appeared that 92% of the population considered that their profile 
retrieved references which they woxxld not have found by their normal 
searching methods, as 258 users ticked one or more of the first three 
advantages (a), (b), (c)* 

Disadvantages : Two users stated three additional disadvantages** 

USERS ^ OPINIONS OF INDIVIDUAL ASPECTS OF SERVICE PERFORMANCE 
(^estions 2-6) 

Questions 2-6 were designed to ascertain the users' opinions of 
individual aspects of service performance* 

Questions 2, 3 and k referred directly to characteristics which 
had already been listed as potential disadvantages in question 1 (options 
j, k, m)» However, in questions 2 - U the user was being asked whether 
he considered that the service was deficient with respect to particular 
performance characteristics (precision, recall ^ coverage) « In question 
1 (j, k, m) he had been asked whether each of these characteristics 
constituted an important disadvantage* !I!he replies to questions 2 ^ k 
therefore provided a more detailed picture of a user's views than would 
be offered by question 1 alone* 

A comparison of replies to question 1 with those to questions 2 - ^ 
also provides a test of the consistency of user replies. Ve assxame that 
a user who mentions a particular deficiency as an important disadvantage 
must, if he is being consistent, recognise the existence of that 
deficiency in his replies to questions 2 - For example a user who 
ticks option l(j) would be expected to reply ^yes^ to question 2. 



See Appendix V«2 
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(a) Retrieval Performance of the Profile 

"Do you consider that your profile retrieves too many Irrelevant 
references?" (Question 2) 

TABLE V RETRIEVAL OP TOO MANY IRRELEVANT REFERENCES 



No. of users % 


Yes 




51 


No 


126 


US 


Don't know 


10 


h 


Total 


280 


1D0 



Of the liUi users who replied ^yes^ to question 2, the majority (118, 82%) 
had also mentioned the retrieval of Irrelevant references as an Important 
disadvantage It Is a measure of the consistency of user replies 

that, of the 120 users who mentioned as a disadvantage, 118 also 
replied yes to question 2. 

"Do you consider that your profile misses too maav relevant 
referencec? ^^ (Question 3) 

TABLE V NON-RETRIEVAL OP RELEVANT REFERENCES 



No. of users % 


Yes 


37 


13 


No 


'hi 


^2(.5) 


Don't know 


96 




Total 


280 


10C 
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Of the 37 users who replied ^yes^ to this question, 25 (68?/^) had considered 
^missed references* (^l(k)) to be an important disadvantage* Of the 27 
users who mentioned option l(k) as a disadvantage, 25 gave the consistent 
reply to question 3* 

(b) Coverage of BA Previews 

"Do you consider that the .journal coverage of BA Previews for 
your reseeurch interests is:-" (Question k) 



TABLE V JOURNAL COVERAGE 



No. of users % 


Adequate 


203 


72(.5) 


Inadequate 


1*6 


16 


Don't know 


31 


11 


Total 


280 


100 



Of the U6 users who considered that the journal coverage was inadequate, 
2k (52^) had mentioned this as a disadvantage. Two users gave inconsistent 
replies # 



(c) Relevance Prediction 

"Do you find it difficult to decide t from the title alone whether 
an article is relevant to your research interests? " (Question 5) 
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TABLE V P. DIFPICULTy IN PREDICTING RELEVANCE 



No. of users % 


Frequently 




15 


Occasionally 




66 


No 




19 


Don't know 


0 


0 


Total 


280 


100 



The majority of users (81%) experienced at least occasional difficulty 
in predicting the relevance of an article, though this was a frequent 
problem for only 15% of the population* 

The users* opinions can be compared with the measurements of relevance 
predictability whi",ii are described in Chapter !+♦ These showed that there 
is an 18% probability that a user will not be able to pretUct correctly 
the relevanc^e of the document on the basis of the enriched title ♦ 

Replies to question 5 vere compared with replies to question l(r)(i): 
"Thore is not enough information on the printout: no austracts". Of the 
75 users who considered the absence f abstracts to be an importa^^t 
disadvantage, 20 reported freqi..mt difficulty in predicting the relevance 

of an article from its tivle, and i+9 reported occasional difficulty* The 

2 

X - test showed a very strong correlation (a -0*002) between a positive 
reply to Q l(r)(i) and the reply ^frequently* to question 5* 

(d) Profile Construction 

"The use of a liaison scientist to provide as;3istance in the 

construct :^ on and amendment of profiles is expensive* Furthermore * 

some search workers may not wish to delegate? this task to 

someone else but would prefer to construct their own profiles * 

Would you prefer : 

(a) To construct your own profile, after receiving the necessary 

instruction on the technique of profile constructiont and have it 
O 

ERIC checked * 



- 82 - 



or (t>) To have the profile construe tod by a liaison scientist . 
as at present? " (Question 6) 

One hundred and fifty eight users {So%) said they would prefer to 
have their profile constructed by a liaison scientist, the remaining 
122 users {hU/o) would prefer to constmct their own profile. 

Replies to this question were compared with those to Question 1 (q), 
"The coding of the profile is difficult to understand". Only 30 users 
were concerned that they coulrL not understand the coding, and 18 of these 
said they would be interested in constructing their own profiles. 

5. USE MADE OF RELfJVANT REFERENCES FOUND BY THE SERVICE 
(Questions 3 and 4) 

"After seleci ong the relevant references on your computer printout , 
do you aagseo uently look up the original articles, provided you have 
notvacfready seen them? " (Question 8) 



TABLE V G. LOOKING UP OF ORIGINAL ARTICLES 





No* of users 




Always 


ho 


17 


Frequently 


'hh 


51 


Occacionaiiy 




30 


Never 


3 


1 


Total 




100 



It appecUo that 1 '2 li^err; "»^) iwfike a habit of Jookin/v up the original 
article:: arid are, therefore, nia^r' : practical u.;o of the printout. 
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"Have you eyperiencei difficulty in obtaining the original artlclea? " 
(Question 9) 

TABLE V H, DIFFICULTY IN OBTAINING ORIGINAL ARTICLES 



No. of users % 


Always 


1 




Frequently 


19 


7 


Occaaionally 


167 


60 


Never 


92 


33 


Total 


260 


100 



The answerc to this question are not merely a reflection of tixe 
available library facilities: users varied in their interpretation of 
"difficulty'*. Thus, some users considered that they had ''experienced 
difficulty in obtaining articles" if their library failed to satlsfj^ 
an inter-library loan request, whilst others considered an ax*ticle 
was diTficult to obtain if it was not available in their own ibrary, 
irrespective of whether or not it cdild be obtained from an outside 
source. 

6. IMPACT OF TtlE BA PREVIEWS SERVICE ON THE USAGE OF CURRM 
PRIMARY. AND SECONDARY JOURNALS (Question;:? 7 and IO) 

(a) Impact on \ii^a^T,e of the printed Biolo^^ucal Abstracts 

"Si nce you stai'ted rece iv ing BA Previews computer printouts , 
has there been an increase or decrease- in your use of the 
printed Biolo/:ical Ab;jtraots for ourrent avaieness and/or 
re i ro do c t i ve ce arch 1 n/r? " (Queijtion '/) 
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TABLE V I. USAGE OP THE PRINTED BIOLOGICAL ABSTRACTS 



Current Awareness 



No* of users % 



Retrospective 



No, of users % 



Increase 


2k 


9 


Decrease 


52 


19 


No change 


6k 


23 


DonH know 


5 


2 


Never use it 


135 





Total 



- 280 



100 



21 


7(.5) 


18 


6 


116 


k^ 


12 


h 


113 


ko 



280 



100 



Participation in the experiment had led to a change in usage of 
the printed Biological Abstracts for current awareness purposes fcr 76 
usvrs, aziii for retrospective searching for 39 users • The usage of the 
printed 3l olo^ica3> Abstracts for current awareness and/or retrospective 
searching increased for 3^ users and decreased for ^6 lasers. Thus, 
participation in the experiment did not result in any mtgor overall 
change in usag'e. 

Subsequent discussion with users revealed that increased usage 
fur current awareness purposes usually resulted from a favourable 
reaction to their* printout, which led to a desiro to consult the 
abstracts, whilst for rf*trospective searches it sirose from a 
realisation that BiolOfrical Abstracts cov2red t^^eir research interests 
more adequately than they had hithorto believed. Decreased usage was 
the result of users finding that they could rrly on their printout to 
retrieve a high proportion of relevant references in their field ^nd 
that manual ccanning of Bi ological Abstracts vas therefore no longer 
necessary. 

It will be noted that the proportion of user? (UO/^o) who said they 
did not use Biological Abstracts for current awareness purposes was 
lower than that for the first questionnaire {oo/o) 11). This 
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apparent anomaly is largely accounted for by the fact that 28 of the users 
who replied that there had been 'no change' and 11 of those who replied 
that there had been a 'decrease' in usage, had replied 'never ixse it' in 
the first questionnaire* 

(b) Impact on general literature searching habits 

Some P€u:ticipants have asked if there will be a peimanent 
BA Previews current awareness service. If there were such 
a service do you consider that you would be able to reduce 
the time spent : 

(a) Scanning current .journals 

(b) Scanning secondsury .journals ," (Question 10) 

TABLE V J. REDUCTION IN TIME SPENT SCANNING JOURNALS 





Current journals 


Secondary journals 




No. of users 


% 


No. of users % 


Considerably 


o6 


31 


136 




Slightly 


91 


32(.5) 


72 


26 


No 


96 


3h 


59 


21 


Don't know 


7 


2(.5) 


13 


s 


Total 


280 


^00 


280 


100 



Further analysis showed that only ^hf/o of the population (i|0 users) 
believed that a regular BA Previews service would not enable them to 
reduce the time spent in scanning either the current primary journals 
or secondary journals. 

Replies to this question were compared with those to question l(e): 
*The service is time-saving*. Of 109 users who considered the time- 
saving feature to be an important advantage ♦ 95 (87;0 said they could 
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reduce the time spent scanning current journals. 

Replies to question 10 were also correlated with those to 
question 1 (l) which asked whether the time-la^ before the appearance 
of an article in 3A Previews was an important disadvantage, V/e 
postulated that a disinclination to reduce the scanning of current 
journals would be associated with a concern about the time-lag. There 
should therefore be a strong correlation between a negative reply to 
question 10 (a) and a positive reply to question 1 (l). We would 
not, however, expect any similar correlation with the expectation 
of reduced usage of secondary journals (QIO (b)) since these would 
probably not be any more current than BA Previews itself. 

The results of the comparison are displayed in Table V K. As 
expected there is no correlation between replies to question 10 (b) 
and 1 (l). However, the expected correlation between replies to 
questions 10 (a) and 1 (l) is weak (a 0.0?). We must conclude 
that the users decision not to reduce his scanning of primary jjumals 
must be influenced by other factors than concern about the lack of 
currency of the S.D.I, service. 
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TABLE V K. CORRELATION BETWEEN REDUCED SCANNING OP 
JOURNALS AND CONCERN ABOUT TIME-LAG 



Reply to Question 10 


Yo who mentioned 
time-lag(Q1 (l))* 


a - significance 
level 


10 (a)« current .ioumals 








users who would not reduce 
scanning 

other users 




h3 


0.07 


10 (b)« secondary .ioumals 








users who would not reduce 
scanning 

other users 




51 

140 


0.62 


* 

I.e. the proportion of users 


giving 


a particular reply to question 10 


who also mentioned time-lag 


as a disadvanta^.;;e, e.g. S^-'o of users who 


would not reduce their scanning of current journals had mentioned 


time-lag as a disadvantage. 









7. INTEREST IN RETROSPECTIVE liJEARCH SERVICE ((^aestion 11) 

^'If ♦ in addition to the current awareness service, it became 
p jsible to operate computer searches of Biolo>':ical Abstracts 
for the last 1-5 year^, would this retrospective search service 
bo of .greater or leGoor value to you thcin the existing: current 
awareness servige? '^ (y^aostion 11 ) 
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TABLE V L, COMPARATIVE VALUES OP RETROSPEJTIVE 
SERVICE AND CURRENT AWARENESS SERVICE 



Vsdue of retrospective service 


No. of users 


% 


Greater vsdue 


117 


U2 


Same vsdue 


71 


25 


Lesser value 


71 


25 


Don't know 


21 


7(.5) 


Total 


260 


100 



There was obviously considerable interest in a service offering 
retrospective searches as 67% of the users considered they would 
find this of equfiil or greater vsdue than the current awsa?eness service* 

IMTEREST IN A PERMANENT CURHENT AWABENESS SERVICE 
(Questions 12 and 13) 

It was important to discover the degree of interest in a permanent 
current awareness service, should one become available. Not only was 
information required concerning the general interest in a regular 
service, irrespective of cost, but also the reaction to the likely 
subscription rate for full cost- recovery service • 

"If there were a permanent BA Previews current awareness service 
would you wish to become a subscriber, assuming that your department/ 
institution would pay the subscription? " (Question 12) 



The replies are summarised in Table V M. 
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TABLE V M. INTEREST IN A PERMANENT CURRENT 
AWARENESS SERVICE (Question 12) 





No. of users 


% 


Yes 


19U 


o9 


No 


hi 


17 


Don't know 


39 


1U 


To tad 


280 


100 



'^If a permanent BA Previevs current awareness service were 
established in the JJ.K.^ on a full cost-recovery basist the 
minlmxim cost of a profile is likely to be £S0-£75^ per annum 
(though improvements in search programs might reduce this 
substantially durinjg the next few years) • Do you consider 
your present BA Previews profile to be worth £^0-£7^ per 
annum? " (Question 



TABLE V N. BA PREVIEWS SERVICE WORTH 

£50-£75 PER mm (Question I3) 





No. of users 


% 


Yes 


113 


ho 


No 


109 


39 


Don't know 


58 


21 


Total 


280 


100 
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It was found that of the 19U users who said ''yes'' to question 12, only 
10U iSk/o) considered that their profile was worth £50-£75 per anniim. 

An. analysis of the employment categories of the users who replied"yes" 
to question 12 showed that the academic and government workers were more 
interested in a pemanent service (7I/0 and 7U/0 respectively) than their 
industrial counterparts (50?^) • However, users who considered the service 
worth £50-£75 was distributed among the three major employment groups in 
a significantly different manner. Only }Qh/o of academics considered the 
service to be worth £SO-£75 compared with U9/o of government and khY<> of 
industrial biologists* Discussion at interviews showed that replies to 
question 13 were influenced by the users/ views of the likelihood of the 
necessary funds being made available. Thus, the comparatively low figure 
for academic work-^rs is explained by the fact that many of these users 
considered that their department would be unlikely to pay the subscription. 
On the other hand, a substantial majority of government workers were located 
at ARC establistjnents and were hopeful that the ARC would make provision 
for subscriptions to mechanised services. 

9. FACTORS INFLU3NCING THE USER^S OVERALL JUDGEMENT OF THE VALUE OF 
THE BA PREVIEWS SERVICE 

Replies to questions 12 and 13 were considered as representing a user's 
overall opinion of the BA Previews service ♦ taking into account all aspects 
of its performance. These replies were therefore compaLred with replies t.; 
other questions which were concerned with specific characteristics of the 
service . The more significant correlations are Qiven in Table V 0# A full 
lint of all the correlations tested appears in an Appendix . 

TheoO comparisons revealed a pattern of factors which apparently 
influenced the user* 3 wish to subscribe to a permanent ser^/ice (Question 12) 

r hio -.^pinion that the service was worth £50-£7^ (Question 13) but, ar, 
expected^ no oingle characteristic had an overwhelming influence on the 
user's jverall jud(:ement -f the service. It appeared that the fact that 
the US'-;:; menti.'ned advantai^TOG of the ser\^icr was i hotter indicator of his 
reaction than his reco^jnition of specific iis advantages. 



See Appendix V.3 
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BEST COPY AVAIUBLE 



TABLE V 0. SIGNIPICMT COREELATIONS BETWEEN REPLIES TO QUESTIONS 12 
13 AND REPLIES TO OTHER QUESTIONS 





Question 12 


Question I3 


Question 


Yes 


No a 


Yes 


No 0 






¥ 


% 


0/ 
70 


Advantages 










Q«1(a) Major value references 


60 


21 0.0001 






Q.1(b) Background references 




U.U(J3 




68 0 . 3I1 


Q.1(e) Time-saving 


] 7 


u. uuu 1 




^9 0.0001 


Q.1(f) Convenient 


22 


1 0.0001 


25 


10 0.001 


Q«1(fir) gives me confidence" 


1:0 


12 0.0001 




21 0.0001 


Disadvantages 










Q.l(j) Too many irrelevant references 


39 


52 0.05 


32 


50 0.003 


O.lfl^ Time-lae* 


.'.5 


52 0.30 


39 


53 0.03 


Q.2 .Yesj Too many irrelevant 
references 


L6 


61; 0.01 


39 


60 0.001 


Q.10(a) or (b) "Considerably] Reduce 
scanning 


70 


33 0.0001 


73 


h9 0.0001 


Notes 










(i) The percentages indicate the proportion of those users giving a 


particular reply to Questions 12 or I3 who also gave 


the 


reply 


indicated in the first column. E.g. the 


first entiy 


in the column 


headed ''Question 12 yes^* indicates that 1 


yO) of users 


replying "yes" 


to Question 12 had also ticked advanta^Te 


1(a). 






(ii) a - significance level* 
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a) Advantages 

Strong correlations were found between replies to Questione 12 and 
13 and mention of the following advantages :- 

(i) Retrieval of MAJOR VALUE references (1(a)) 

(ii) Time-saving (1(e)) 

(iii) Convenient (1(f)) 

(iv) .... the service gives me confidence (1(g)) 

Users who mentioned "retrieval of background references" (1(b)) as an 
advantage were more likely to reply "yes" to Question 12 but this advantage 
did not appear to be related to their judgement of the "worth" of the 
service (Question I3). 

The significance of "time-saving" (1(e)) is further emphasised by replies 
to Question 10, which showed that 70% of the users who answered "yes" to 
Question 12, as compared to 33% of those who replied "no", expected that 
the service would enable a considerable reduction to be made in the time 
spent scanning either the current journals or secondary journals. 

b) Disadvantages 

The retrieval of "too many irrelevant references" (l(j);2) was the 
only disadvantage which showed a significant coi-relation with the user»s 
overall judgement of the service. 

Users who were concerned about the time-lag (l(l)) tended to reply in 
the negative to Question I3 but the corresponding correlation with Queotion 
12 was not significant. 

Further disadvantages of the service, i. "mioses too many relevant 
references" (1(k)); inadequate coverage (l(m );!,); the provision of 
references of little practical value (l(n)); the unattractive computer 
prirtout (l(o)); and the absence from the printout of abstracts (l(r)-i), 
author addresses (-ii), or the language of the original article (-iii), did 
not appear to exert any significant influence on the user's assessment of 
the vedue of the seivice. 
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c) Use made of the aervice 

It is of interest that users who looked up the original articles 
(Question 8), and were presumably making the most practical use of the 
service, were no more likely to reply "yes" to Questions 12 and 13 than 
those who did not*. 

10. RELATI ONSHIP BETWEEN USERS' OPINIONS AND QUANTITATIVE PERFORMANCE 
MEASURES 

Replies to questions 12 and 13 were compared with individual values 
for the performance measures, i.e. output size, % precision, % recall, 
% coverage, % novelty. Ve wished to determine whether user satisfaction 
with the service, as expressed in positive replies to questions 12 and 13, 
could be related to 'good' values for a particular performance parameter, 
e.g. whether users who were interested in a permanent BA Previews service 
had profiles witii above average recall. 

Tiie results of correlation tasts, using Wilcoxon's U-statistic, are 
summarised in Table VP which shows, for each parameter, mean values for 
those who replied yes and no (cr don't know) to question 12 and similarly 
for question I3. 



TABLE V P CORRELATIONS BETWEEN PERFORMANCE 
MEASURES AND USER SATISFACTION 



Parameter 


Reply to question 12 


a 


Reply to question I3 


a 


Y 


N 


Y 


N 


Output size 
(refs/issue) 


3U.0 


39.9 


(0.79) 


36.3 


35. li 


{0.h2) 


% Precision 


39.8 


38.2 


(0.56) 


i+1.2 


37.8 


(0.1+3) 


% Recall"^ 


61.5 


61.6 


♦ 


63.3 


62.0 


* 


% Coverage* 


69.3 


79.5 


♦ 


71.. 5 


77.2 




% Novelty 


79.6 


69.3 


0.001 


79.2 


73.7 


0.006 


* Values for 0 were not computed but are assvuned to be non 
(>> 0.05) on the basis of the mean values given. 

""^ BV-6 samples. 


-significant 



Details of significance tests appear in Appendix V3. 
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Only the novelty ratio shows significant correlations with questions 
12 (o - 0.001) and 13 (u ^ 0.006). This parameter is a rather unsatisfactoiy 
measure of service perfoimance because of its composite character (cf p. 55 ). 
Nevertheless it is the best predictor of user satisfaction among the 
parameters which were measured. 

11. SHARDIG THE PRINTOUT 

At the commencement of the experiment .'7 users (ll;?c) said that they 
expected to share their printout with collea^jues, as a result of which 35 
Group Profiles and 12 Shared Profiles were written (Chapter II). It was 
anticipated that general satisfaction on the part of many users would have 
led to increased sharing of their printout. Accordingly, users were asked: 

"How many people ( including yourself) see your printout regularly? " (Q.IL) 

It was found that 10? users (38%) wero sharing their printout with 
one or more colleagues at the time of the final interviews. 

12. FORM OF COMPUTER PRINTOUTt CARD OUTPUT 

During the course of the experiment, users showed considerable interest 
in the possibility of the computer printout beirxg available on cards as 
opposed to paper. It was considered important to iind out what opinion 
the total population of users had of this card output. 

In order to demonstrate the alternative format, one issue of 
BA Previews (52 (23)) was printed on cards. Accompanying the printout 
was a short questionnaire on which users were asked to tick one of three 
options: 

1* I prefer the card output 

2. I prefer the paper output 

3. I have no preference for cards or paper 

There was an overwhelminie: preference (95-0 for the card output* 



Appendix V.|. 
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CHAPTER VI 
INFORMATION CEMTRES 

1. INTRODUCTION 

In Chapter III reference was made to eight information centres which 
agreed to take part in the experiment. They were 

Bee Research Association (Cerrards Cross) 

Biodeterioration Centre (Birmiagham) 

Biomedical Information Centre (Sheffield) 

Commonwealth Bureau of Animal Health (Weybridge) 

Commonwealth Bureau of Helminthology (St. Albans) 

Commonwealth Forestry Bureau (Oxford) 

National Documentation Centre for Sport, Physical Education 
and Recreation (Belfast ) 

Project FAIR: Specialised Information Centre for Biomedical 
Engineering (London) 

These centres represented a special category of participants in that 
they served as clearing houses for the collection and subsequent dissemination 
0^ information to research workers. They were therefore attempting to 
satisfy the information needs of others rather than their own individual 
requirements, as was the case of the ordinary users in the experiment. 

Each centre was allowed up to three profiles. In the case of those 
centres whose interests were cenfined to biology, an attempt was made to 
construct a profile which resembled as closely as possible the contents of 
their information bulletin or abstract journal. Tne range of intereots of 
some centres extended ;o subject fields other than biology and therefore 
BA Previews could not he expected to provide complete coverage. 

In common with other participants in the experiment, three interviews 
were given at each centre in order to construct the initial profiles, to 
amend these profiles, and to discuss the final questionnaire. 



In Judging the relevance of the references on their printout most 
centres four.d that it was impoasible to differentiate between MAJOR VALUE . 
and MINOR VALUE: a reference was either relevant (i.e. selected for 
incorporation in their system) or irrelevant (i.e. not selected). 
2. USER REACTION TO THE SERVICE* 

The results of the questionnaire survey of individual users have been 
described in Chapter V. A modified version of this questionnaire"*" was 
circulated to the eight information centres and, with one exception, these 
were completed and returned prior to the final interview. Prom an analysis 
of the replies given by the seven centres who completed the questionnaire, 
. t^e following information was obtained 

a) Advantage s and disadvantages of the service (Q.I) 

Tables VI A and VI B show the number of centres which considered the 
advantages and disadvantages listed to be of importance to them. In addition 
to those listed, the following disadvantages were given 

(i) "Lack of punctuation in the titles prevents direct incorporation 
of the reference into our publication." 

(ii) "Inadequate name of the source for items from BioResearch Index 
necessitates consulting the original." 

(iii) "It is sometimes difficult to interpret the publication citation, 
especially for non-periodicals." 



* 

The authors* comments on questionnaire replies are given in parentheses, 
^ See Appendix VI. 1 
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TABLE VI A. INFORMATION CENTRES: ADVANTAGES OP 
THE BA PREVIEWS SERVICE 



Advantage of the service 



No. of centres ticking this 



a) The profile retrieves references which 
we might not otherwise locate 

b) The profile retrieves articles from 
journals which we do not cover in our 
existing literature searching 

c) The service retrieves references from 
non-;)ournal publications, e.g. books, 
symposia t conference proceedingstetc. 

d) The service is time-saving, i.e. it 
obviates the need for manual searching 
of the hard copy Biological Abstracts 

e) The service is convenient* i.e. it 
obviates the need for copying out 
references from secondary journals, as 
the printout can be used for processing 
our bulletin/index 



Each of the seven centres ticked more than one advantage 
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BEST COPY AVAlUBli 



TABLE VI B. INFORMATION CENTRES: DISADVANTAGES 
OP THE BA PREVIEWS SERVICE 





Disadvantage of the service No. of centres ticking this* I 



g) The pr6file retrieves too many 
irrelevant references 

h) The profile misses too many relevant 

3 

references 

i) There is too much delay between 

publication of an article and its 4 
appea^'ance in BA Previews 

j) The journal coverage is inadequate 2 

k) The computer printout is unattractive 5 

l) The computer printout is difficult 

to file 1 

m) The coding of the profile is difficult 
to understand 

n) There is not enough information on the 
printout: 

(i) no abstracts 1 

(ii) no author addresses 

(iii) no indication of the language ^ 
of the original paper 



i 



Kach of the seven centres ticked -ore thin one disndvnnta^e 
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b) Service performance (Q.2-4) 

The centres were asked if they considered that their profile(s) 
retrieved too many irrelevant references or missed too many relevant 
references and also if they regarded the journal coverage of BA Previews 
as adequate . Their answers are shown in Table VI C . 



TABLE VI C. INFORMATION CENTRES: OPINIONS OP SERVICE PERFORMANCE 



Question 


No. of Users 


Yes No Don»t know 


Profile retrieves too mariy 
irrelevant references (Q 2) 

Profile misses too many relevant 
references (Q 3) 

Journal coverage of BA Previews 
(«» U) 


3 5 1 

4 3- 


Adequate Inadequate DonH know 


3 4 - 



[it was to be expected that some information centres would consider that 
their profile was missing relevant references ^ as their aim was complete 
coverage of their interests and they were therefore able to cite references 
located from other sources^ which were relevant to their profile but not 
retrieved by it* It was also to be expected that the journal coverage would 
not be comprehensive enough for those centres with multidisciplinary 
interests,] 

c) Relevance prediction (Q#5) 

All the centres said that they encountered some difficulty in deciding, 
from the title alone > whether an article was relevant to their interests , 
five frequently and two occasionally, 

d) Profile construction (Q,6) 

Five centres said that they would prefer to construct their own 
profile t the other two preferring to have it constructed by a liaison 
scientist. 
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e) Impact of the aervioe on the uaage of th e printed B<nlnff<noT 
ADBtracta ^Q.") ' 

Five of the centres never used the printed Biologioal Abstraota . Of 
the two centrea which did uae it, one had decreaa'ed the uaage for current 
awareneaa puxpoaea; the other reported "No change." 

[The decreaaed usage resulted from a realisation that the printed Biological 
^'^'^^^ ^ l>e Bcanned, an operation which hitherto had taken 

about an hour per issue. It only took ten minutes to scan the printout, 
which retrieved relevant references which ml^^t have been missed by manual 
scanning of the printed version]. 

^) Inipact of the service on liter ature aoaiminff bv the oentrn (Q.8) 

Only one centre considered that a regular BA Previews service would 
enable it to reduce slightly the time spent scanning current Journals. 
Three centres considered they would be able to reduce sli^tly the time 
spent scanning the secondazy journals. The other three centres stated that 
no reduction in scanning time would result. 

e) Hetrospec tive search service (Q.9) 

There was little interest in a retrospective search service. Only one 
centre said that it would be of the same value to them as the existing 
current awareness service and none considered it of greater value. 

^) Current awareness aervtoft (Q. 10-11) 

Pour centrea said that, assuming adequate funds were available, they 
would wish to subscribe to a permanent ^ Previews service, but only two of 
these considered their profile to be worth £50-£75 per annum. The remaining 
three were uncertain of their interest, though one of them considered their 
profile to be worth the full cost-recoveiy rate. 

i) Usage made of BA Previews printout (Q.I 2) 

At the final interviews ttie centres were asked for Information on the 
use they had been making of their printout during the experiment and their 
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general interest in a permanent BA Previewa service, should one be 
established. Their comments were as follows 

CEMTRE 1 ; This centre had three profiles, each of which attempted to 
cover the same subject area as one of three published information bulletins. 
Some of the relevant references from the printout were incorporated in 
these three bulletins, which were distributed to 500, 400 and 150 sub- 
scribers, respectively. The BA Preview s service provided supplementary 
material, which would not have been located from their other sources, but 
the poor timeliness of the references prevented the service replacing 
existing scanning of current jovimals and of Current Contents . Nevertheless, 
it had proved sufficiently valuable to justify a subscription. 

CEWTRE 2 ; This centre had three profiles designed to cover very specific 
topics. Some of the relevant references were incorporated in an information 
bulletin which was distributed free to 250 workers and a few other 
references were incorporated in the centre's in-house system. The centre 
was interested in purchasing one or more profiles, but would not be prepared 
to pay the full rate unless the cost could be shared with another user, i.e. 
a small group could be generated. They considered that "the viability of a 
mechanised service such as BA Previews stands or falls on the success of 
creating groups." 

CENTRE 3 ? This centre had one profile designed to cover the biological 
aspects of its multidisciplinaiy interests. Most of the relevant references 
were incorporated into a published information bulletin, which was 
distributed to 500 subscribers. It was estimated that the printout retrieved 
"approximately 300 references/annum which were not previously known." The 
centre calculated that, assuming a subscription of £60/annum, the cost of 
each reference would be 20p, which was more than that for Current Contents 
(7p) but much less than that for their "Cooperating Specialist scheme" (50p). 
BA Previews could not "replace our other sources, because of the delay 
factor and must therefore be regarded as an ancillary source, producing at 
present levels 8^9^ of our total input. The interest in a subscription 



To preserve anonymity the centres are referred to by numbers; these 
numbers do not correspond to the order in which the centres are listed 
on page 9$. 
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will depend upon the value put upon this supplementary material." 

CEMTRE 4: This centre had one profile designed to cover the entire range 
of its interests. Some of the relevant references were incorporated in a 
published abstracting journal, which was distributed to 1,500 subscribers. 
The centre was interested in subscribing to BA Previews , but was doubtful 
about the availability of funds. In order to justify purchase of a profile, 
the centre would have to estimate hon» many of the relevant references 
retrieved by the profile would not have been retrieved by existing methods, 
i.e. scs^ining of current joxirnals and alerting done by contacts in foreign 
countries . 

CENTRE 5t The profile for this centre attempted to cover the same subject 
area as the contents of their abstracting journal. Some of the relevant 
references were incorporated in this journal, which was distributed to 
1200-1500 subscribers. They estimated that 10^ of the relevant references 
retrieved by the profile would not have been found by th'iir existing 
methods; however, some of these would not be selected for inclusion. They 
were uncertain of their future interest in a BA Previews service and would 
find it difficult to justify the full cost-recovery rate. 

CENTRE 6: This centre had one profile which covered a very broad subject 
area. Although producing much irrelevant material, the profile retrieved a 
considerable number of references which would have been missed by their 
selective manual searching of the hard copy Biological Abstracts . An 
abstracting journal and an indexing journal were produced by the centre and 
these were distributed to 2,000 and 500 subscribers, respectively. It w^.s 
estimated that 5 references/printout were incorporated in the abstracting 
journal and 15 references/printout were incorporated in the indexing journal. 
Very few articles of any value were found in BioResearch Index . This 
centre would be interested in subscribing to a permanent service nnd would 
hope to be able to justify the cost. 

CSJiTRE 7; The profile attempted to cover the biolof'ical nspectc of their 
ringe of interests. Only a few of the relevant references retrreved by the 
printout were incorporated ir. their abstracting journ-il, which was distributed 
to 1,190 subscriber::. They said "We have trcited the project by and large 
us nn experimental, not a practical, exercise. V/e have used it to borrow 
one or two articles of peripheral interest that we .-nipht not otherwise have 
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seen." They therefore showed little interest in a permanent service, as 
they preferred to rely on their existin^f methods. 

CEyTRE 8 1 No questionnaire returned. 

3. SEH/ICE PEBFOBMANCE 
a) Precision ratio 

The precision ratio (MAJOR -t- mOR VALUE items) was estimated for each 
of the eifi^t infor^tion centres from relevance judgements jiven on feedback 
cards. Both the average output size and the precision figures were based 
on data from six issues 01' BA Previews , four corresponding to Biological 
Abstracts and two to BioResearch Index . The results are shown in Table VI D, 



TABLE VI B. IMFORNATIOK CE3TTRES: PRECISION RATIOS 



Inf oination Centre 


Output Size* 


Precision Ratio 
% 


Centre 1 


8J^ 


33+ 


Centre 2 


Sh 


21 


Centre 3 


73 




Centre 


36 


1*5 


Centre 5 


28 


87 


Centre 6 


76 


11* 


Centre 7 


1J+8 


66 


Centre 8 




18 


Average 


86 


39 



Average number of references/issue 
* This figure is the average for 3 profiles 
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b) Novelty ratio 

Novelty ratios were computed for the five information oentxes who took 
part in the evaluation phase of the experiment and provided data on the 
proportion of relevant references retrieved by their profile which they had 
not already seen. The results are given in Table VI E. 



TiBLE VI E. INPOHMiTIQN CBlTHESt NOVELIT RATIOS 



Information Centre 


Novelty Batio {%) 


M 


BBI 


Bi/BRI* 


Cwitre 1 


69+ 


81,+ 


77+ 


' Centre 3 


35 


81 


1.7 


Centre $ 


33 


82 


ill 


Centre 6 


62 


li7 


Sh 


Centre 7 


16 


1? 


16 


Hean 


US 


63 


1.9 



« 

Prom li issues of BA and 2 of BRI (cf . Chapter IV) 



+ Mean value for two profiles 

As in the case of individual users, BioResearch Index (BRi) supp. ,u a 
much hi^^er proportion of references not already known to the centr© than 

Biological AbstraotB (ba). Novelty ratios showed wide variations among 
the individual centres but, with one exception, were substantially lower 
than the average value (7796) for individual users, as would be expected. 
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CHAPTER YII 
CQHCLPSIOaiS iMD STOMAHr OP HE8ULTS 

1* IHTOQDPCTIQW 

a) Ob.leoti™ 

It will be Appaxent that the project achieved its first objective of 
developing an ezperiaental SDI service based on the computer searching of 
BA Previews, and thereby introducing a large population of biologists to a 
■echanised service, in aost cases, for the first time. The other objectives 
defined in Chapter I were: to investigate methods of formulating user 
profiles for searching the BA Previews data base; to assess user reactions 
to the BA Previews service; to obtain quantitative measures of performance; 
and to draw conclusions as to the demaxid among biologists for a mechanised S.D.I, 
service and the suitability of BA Previews as the basis for such a service. 
Secondary objectives were the collection of information on literature usage 
by biologists, and an assessment of tho interest in a retrospective search 
service. 

Before suomarising the more important results obtained and the conclusions 
which can be drawn in respect to these objectives, some general observations ^ 
are necessary oni (a) the choice of the user population, and (b) the role of 
the liaison scientist in the project. 

b) Pie User Population 

TbB results obtained in relation to each of the objectives listed above 
are ^plicable to the biological comcnmi'ty as a whole only to the extent to 
which participants in the project formed a representative e3q>erimental 
population. Figures obtained from a s\irvey of the employment type and subject 
interests of biologists were used as a guide in the selection of participants. 
This method of selection ensured that the final population of 337 users was 
reasonably representative of the major employment categories and the major 
subject groupings of biologists. It is assumed that the 151 users who 
participated in the evaluation were a random subset of the full population. 

c) The Liaison Scientist 

The \ise of * liaison scientists' to maintain contact between the user and 
the service and, in particular, the emphasis on personal interviews with 
participants, was characteristic pf the jg^ Previews Project and also of other 
projects undertaken by the Eacperimental Infonnation TJhit^^* 
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The amount of effort which was devoted to interviewing users, to the 
definition of their information needs, and to the f ovulation and maintenance 
of profiles, was almost certainly greater than would be feasible in a 
permanent cost-recoveiy service. The intensive effort devoted to each user ? 
is very desirable during the development and evaluation of a new, experimental 
service, and was probably a significant factor in ensuring the continuance of 
user interest and co-operation throwi^out the two years of the project. It 
was not possible to assess whether the reduced level of assistance, which 
would be offered to users by the supplier of a permanent cost-recovery service, 
would result in a significant reduction in the quality of the search profiles. 

It seems probable that, in any future permanent service, either the user 
would have to take greater responsibility for the construction of hia own 
profile, or the liaison function wo\ild have to be provided by a local 
information officer or librarian. In either case the provision of effective 
training in r«thods of profile construction for either the user or Ihe 
information officer would be essential. 

2. METHODS OF PROFILE CQNSaTOTCTION 

^ The BA Previews data base contains both natural language (the enriched 
titles) and controlled language (CROSS code and BioSystematic) Indexes. Much 
of the first year of the project was devoted to gaining experience in the 
use of these indexes in the construction of user profiles. Special studies 
were undertaken by the liaison scientists of the BIOSIS indexing philosophy 
in selected subject areas as it applied to enrichment terms, BioSystematic 
codes, and CROSS codes. 

Many of the profiles written in the early stages of the project underwent 
major amendments as our knowledge of the data base and o\ir methods of >,'ofile 
construction improved. The final versions of the search profiles made 
extensive use of all three indexes. All but five profiles contained word 
terms, and the CROSS and BioSystematic codes were used in 89% and 76% of 
profiles respectively. A comparison of profiles written by the three liaison 
scientists showed dramatic variations both in the number of terms and in the 
complexity of the profile logic, but these variations were not reflected in 
any substantial differences in the retrieval performance of the profiles. 

Qirichment Terms; these are guided by BIOSIS indexers in order to make the 
titles more meaningftd, ^d their presence undoubtedly enhances the 
effectiveness of word searches. A small scale study of 150 references, all 
of which had been retrieved by word terms alone, ahowed that 31% would not 
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have been retrieved without the presence of the enrichment terns. However, 
it oust always be kept in mind that the enrichment terns are a natural- 
laagua^^ index and that allowance must be made for the use o£ alteinative 
grammatical forms, synonyms and related teims* A special study war made of 
the potential value of enrichments for specifying geographical locations. 
This type of enrichment could be especially useful for users interested, for 
example, in environmental biology. However, it was found that the enrichment 
texms were not assigned with sufficient consistency to provide a reliable 
method of retrieving articles which were concerned with specific geogrcqphical 
locations. 

BioSystematic Codes : a study of BIOSIS indexing practice for selected 
BioSystematic codes showed that these codes are applied with a high degree 
of consistency. This is reflected by the fact that taxonomists were among 
the most satisfied users of the service. The code for Hominidae (SYS85215) 
was found to be especially useful in restricting a search output to articles 
conoexned with * humans*. 'Jhis BioSystematic code was always preferred to 
the use of the many synonyms for *};umans* (man, adolescent, girl, etc). 

CROSS Codes : although there is no doubt as to the value of using these 
controlled language subject inde:^ texms in search profiles, several problems 
were encountered in their use: 

(i) the CROSS code listingi^^ supplied by BIOSIS, is in code number order. 
It was necessary to prepare a CROSS code list in alphabetical order of 
subject as an aid to the liaison scientist in selecting the codes to be 
used in the search profile. 

(ii) the CROSS codes correspond to relatively broad subject areas. Only 
CROSS codes are available to cover the subject matter of the biological 

sciences. In contrast. Index Medicus uses approximately 7fOOO separate 
subject headings for indexing the medical literature. In most cases, 
therefore, CROSS codes can only be used in coordinate searches involving 
the linkage either of two or more CROSS codes, or of CROSS codes with other 
term types (e.g. title words). 

(iii) the detailed instructions provided to the BIOSIS indexers^-^'^^ are 
not available to ]BA Previews subscribers, and difficulties were encountered 
in recognising the subject area corresponding to some of the codes. 

(iv) the subject headings of the CROSS code index are revised annually; 
some new headings are added and some existing headings are deleted or 
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r«-deflned. All profiles must therefore be checked to discover whether 
•nendments are needed as a result of changes in the CROrs index. 
Ifeif ortunately, tho;j«h the changes are implemented in the first BA Previeura 
issue of each year, immediate revision of the profile is not possible as the 
••Guide to BA Previews"^^\ which gives details of these changes, is not 
received by subscribers until March or April. 

3. LITERATUBE SEARGHINfl HAT^ITS OP BIOLOGISTS 

The most striking characteristic of literature usage by biologists, which 
was revealed by the initial lueetionnaire survey, was the great diversity of 
both the primary and the st^coidaxy sources which are used by the biological 
community. In all, 698 different primary Journals were seen regularly by one 
or more of the 333 participants who completed the initial questionnaire. Only 
66 of these journals were mentioned by ten or more users. The participants 
also cited 150 different secondary publications covering specialised subject 
areas (e.g. iiTal.-? Pollution Abstracts), but only ten of these were mentioned 
by ten or more users. 

Of the secondary publications which cover much broader subject areas, 
those most commonly used for current awareness searching were Current Contents 
(157 twers, kl%), Biological Abstraota (105, 32%) and Chemical AbBtraots 
(i+O, 129^). Only a minority of participants (2290 had previous experience of 
mechanised S.D.I. or retrospective search services and this experience most 
conmonly took the form of a MSDLAES retrospective search. 

The questionnaire replies provided detailed information regarding usage 
of the printed Biological Abstracts . It was found that only 178 {SWi) of 
those who had volunteered to take part in the experiment had previously made 
any use of the printed publication for either current awareness or retrospective 
searches. Of the who did not use it, 68 belonged to organisations which 
had no subscription. Of the 10^ participants who used it for current awareness 
searches, only 33 claimed to use it regularly. 

The most popular method of searching Biological Abstracts was item-by- 
item scanning of selected subject sections. This procedure was followed by 
68?^ of those who used the journal for current awareness searches and by 61^ 
of those who used it for retrospective searches. Though the majority also 
made some use of at least one of the four indexes (author, BASIC, 
BioSystematic, CROSS) there was evidence of substantial under-use of the 
printed indexes. In particular, only ten participants (6?^ of those who 
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used Biological Abstracts^ claimed to make use of the CROSS index. The BASIC 
index, which might have been expected to be extensively used for cxxrrent 
awareness searching, was in fact used in only k9% of current awareness searches. 
In discussions with the participants, the impression was gained that a 
significant number of biologists either disliked the printed indexes or did 
not understand how they should be used. 

h* SEHVICE PERFORMANCE AND USER BEACTION 

The quantitative measures of performance, such as recall and precision, 
were based on user judgements as to the relevance of individual articles to 
their interests. The users' own opinions of specific aspects of service 
performance (e.g. precision, appearance of output), and also his overall 
judgement of the quality of the service were provided by his replies to the 
final questionnaire. The interpretation of these replies was often facilitated 
by informal discussions with participants. The measurement of performance 
parameters and the assessment of user reactions are therefore complementary 
aspects of an evaluation of a reference retrieval system. 

The data which were collected in the present experiment defined the 
characteristics of an experimental BA Previews service. The significance 
of this data, as a means of predicting effectiveness and user satisfaction 
for a future permanent operational service, must be qiialified by the fact 
that the experimental service has two characteristics which are unlikely to 
be present in a permanent BA Previews service: 

(i) the use of liaison scientists to ensure intensive personal contact 
with users of the service. Their influence both on user interest and 
on quality of profiles has been discussed above (page 10^. 

(ii) the provision of a service in return for feedback but without any 
charge to the user. There were, therefore, none of the cost restrictions 
on the number of search terms and/or the size of output which are 
characteristic of the price structures of most cost-recoveiy or commercial 
services. 

a) Output size« Recall and Precision 

The mean output size of the 1 142 BA Previews profiles belonging to users 
who participated in the evaluation phase of the experiment and returned the 
necessary feedback was 3h items/issue. The estimated mean precision, i.e. 
the proportion of retrieved references which were judged relevant, was 
38,6-1.99^ Approximately one third of the relevant items were judged by the 
user to be of MAJOR VALUE. 
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An estimate of 62.5^3.2% was obtained for the recall ratio - the 
proportion of relevant items in the data base which were actually retrieved 
by the search profile. As was fovind in the evaliiations of other reference 
retrieval services^ * * % the mean recall for MAJOR value articles {66,h'3.Q%) 
was significantly higher than that for MINOR value articles (55.9-1.1%). 

Our precision and recall estimates are characteristic of the experimental 
service which was offered to users, and not of the data base alone. The 
values obtained are therefore influenced by a number of variables corresponding 
to the different parts of the retrieval system, i.e. the user-system interface, 
the search programs, the design of search strategies, and the data base. In 
particular, some recall and precision failures undoubtedly occurred as a 
result of inadequate communication between the user and the system. Thus, 
the search strategy may never even have attempted to retrieve some types of 
relevant material either because the user gave an incomplete description of 
his needs to the liaison scientist, or because the liaison scientist 
misinterpreted those needs. A detailed analysis of individual retrieval 
failures^^*^*^^^ would have been necessaiy in order to establish the relative 
proportions of precision or recall failures which are attributable to the 
communication process, and to each of the other subsystems which comprise 
the service. 

b) Coverage 

It was estimated that 75.1'-2.1% of relevant articles (from serial 
publications) encoimtered by the user in the co\irse of his routine current 
awareness searching actually appeared in BA Previews, irrespective of whether 
or not the search profile was successful in retrieving these items. Samples 
of relevant references retrieved from three other major data bases were also 
used to estimate relative coverage ratios for three subsets of \isers whose 
interests were arbitrarily classified as biochemical, biomedical and '^core 
biology" respectively. Each subset comprised approximately one third of the 



All our coverage estimates arc for articles from serial publications only. 
However, only the CAC sample contained a si/^ificant proportion (289^) of 
items from non-serial sources (books, government reports). 



ERLC 



experimental populatio--* For participants with some biochemical interests, 
the coverage ratio of BA Preview , relative to CA-Condensates . was $3.2^3.3%. 
l^OT those with a biomedical bias, a coverage ratio of 72* 3-2. 8% relative to 
MEDLAES was obtained, and a sample of references retrieved from Science 
Citation J jidex gave a relative coverage ratio of Q3.S^h^Q% for the "core'* 
biologists* • 

The majority of the articles which were not covered by BA Previews were 
from jotimals which were scanned by BIOSIS, but the individual articles had 
been excluded from the data base, presumably on the grounds of quality or 
becaixse they were judged to be outside the subject area of the data base« 
With the exception of the CAP sample, over 809^ of 'coverage fail\ires' were 
of this type« 

The CAC samples gave a lower relative coverage ratio than any of the 
other sa2!qples« This is almost entirely accounted for by the larger number 
of articles in the CAC sample which were from joumals not scanned by BIOSIS* 
It appears that the joimial coverage of BA Previews does not extend as far 
into biochemistry as it does into biomedicine« 

Ve are not entitled to use these estimates to make comparisons between 
BA Previews and other data bases* Such comparisons would be possible only if 
estimates of the coverage of other data bases, relative to BA Previews, were 
also available* However, these results do suggest that a substantial 
proportion of biologists require information from more than one of the 
discipline-oriented services, and they therefore emphasise the need for 
research into the merging and repackaging of the existing magnetic tape data 
bases* 

c) Novelty Ratios 

An estimate of the novelty ratio of BA Previews showed that 76*9il*8% 
of relevant items were not previously known to the user* The corresponding 
estimate for MAJOR VALUE items alone was 57« 2*3*0%* It appears that, 
predictably, research workers are more likely to have prior knowledge of 
MAJOR VALUE than of MIKOR VALUE articles* Evidence is presented in Chapter IV 
that the high value of the novelty ratio is a function of the retrieval 
performance of the BA Previews service, relative to the users ^ existing 
methods, rather than of the relative c\xrrency* The high value of the novelty 
ratio may partly be due to the fact that the users were less conscientious 
in their own manual searching as a res\ilt of receiving the BA Previews service* 
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Howaver, a reduction In manual scanning woidd, in itaelf, be indicative of 
ViBBV confidence in the S.D.I, aezvice. 

d) Overall Beoall 

The novelty ratio of the service must be considered in conjunction with 
the estimated recall and coverage ratios. If the combined effect of reoall 
and coverage performance is considered (the overall recall), it appears that 
the service was retrieving no more than h7% of the relevant material which was 
found by the user by his normal methods. On the other hand the novelty ratio 
(779^) indicates that the user was finding oidy 23% of the relevant material 
retrieved by BA Previews. Ttiaa, to some extent, the Bi Previews service 
complements rather than conpetes with the user*s existing methods (thou^ it 
need not be assumed that retrieval of jOl relevant items is necessaxy for 
complete user satisfaction). 

•) CoBPagative Pi>w>nTm«yi ce of BloBesearoh Index and Biological Abstracts 

Outputs from issues of BioResearoh Index (BRi) and of Biological Abstracts 
(BA) were almost identical with respect to the total number of iteas retrieved 
per issue (33. L and 31.9 respectively) and the proportions both of MAJOR and of 
MINOR value articles in the output. Ihe novelty ratio for relevant items in 
ffil (Bh,Bt^,Q%) was significantly higher than that for BA {73.lr2.1%). 
BioResearoh Index contains items from non-serial publications such as US 
Government reports, books or conference proceedings, and also other material 
not considered suitable for inclusion in Biological Abstracts . The evidence 
presented above indicates that this type of material must not be considered as 
being of secondaxy importance to the user. 

f ) User Reactions to the Service 

The two features of the service which appeared to be most appreciated 

weret 

(i) it saved time by allowing uF>>rs to reduce the time spent in searching 
the literature. This was mention;;.! as an iisportant advantage of the 
service by 109 users (399^ of the population). Furthermore, in the event 
of a permanent BA Previews service being available, 31^ and h9% of users 
expected that they would be able to considerably reduce their scanning of 
current primary Journals and of secondary publications respectively. Only 
liO users {M%) ^ ngt think that a regular ^A Previews service would enable 
them to make at least some reduction in their manual searching. 
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(ll) the service retrieved references which would not othervflse be located* 
This advantage was mentlonedi In one form or anotheri by 92% of the 
popxdatlon* This assessment by the users Is supported by the hlg^ value 
obtained for the novelty ratio (77%) of relevant references* 

The most frequently mentioned disadvantages of the service were: 

(I) the delay between the publication of an article and Its appearance In 
24 Previews of the users)* 

(II) the retrieval of too many Irrelevant references (li3%)* 

That the profile missed too many relevant references, or that the coverage 
was Inadequate, were mentioned as major disadvantages of the service by only 
9% and 10% of the users respectively* 

A niomber of other disadvantages of the service, which did not relate to 
Its retrieval performance or currency, were each mentioned by significant 
minorities: the difficulty of filing the computer printout (39%)f 
Indication given of the language of the original paper (37%)f the imattractlve 
appearance of the computer printout (28%), and the absence of author addresses 
(27%) or of abstracts (27%) from the cooQ)uter printout* Participants also 
expressed an overwhelming preference for card as opposed to paper output* 

It appeared that a high level of user Interest was maintained throu^out 
the project. At the end of two years, 302 of the orLgrinal 353 users were 
still receiving BA Previews printouts and retiiming feedback cards. In 
replies to the final questionnaire 191 users (69%) expressed an interest in 
subscribing to a permanent BA Previews current awareness service, though 
fewer (113) thought that the service was worth £50-£75. Replies to these 
two questions, which can be considered as expressions of the users' overall 
judgement of the quality of the service, were coo^eured with replies to other 
questions relating to specific cheu:acteristics of the service. These 
comparisons revealed a pattern of factors which influenced the user's overall 
assessment, though no single characteristic exerted an overwhelming influence. 

Users who nominated 'time-saving', 'convenient', or 'the retrieval of 
MAJOR VALUE references which woidd not otherwise be located' as important 
advantages, were significantly more likely to express an interest in 
subscribing to a permanent service and to Judge that the service was worth 
£50-£75. Criticism of the retrieval of irrelevant material or, to a limited 
extent, of the lack of currency, tended to be associated with a negative 
reaction to the service. None of the other disadvantages mentioned by 

* TtiiB figure includes eight instances in which the original user had 




transferred his profile to a colleague. 
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partloipants appeared to exert any influence on the user's overall judgement 
of the quality of the service. It can be concluded that these disadvantages 
are considered by the user to be annoyances rather than major drawbacks* 

Correlation of performance measures and questionnaire replies for 
individual users revealed that the novelty ratio was the best indicator of 
user satisfaction. 

Participants appeared more satisfied with their BA Previews profile than 
they had previously been with the printed Biological Abstracts . !Ehis 
preference for the compiter services was related, in part at least, to the 
under-usage of the printed indexes to which reference has already been made. 
In discussions with users, it frequently transpired that research workers had 
been unable to establish an effective method of searching the printed 
Biological Abstracts and were therefore failing to locate potentially 
interesting articles. As a result, some participants were pleasantly surprised 
at the number of relevant references retrieved by the profile. The latter 
offers a much less laborious means of searching the indexes and, in particular, 
the CROSS index which was rarely used for manual searches. 

g) Interest in a Retrospective Search Service 

Questionnaire replies revealed a considerable interest in a retrospective 
search service} U2% of participants considered that it would be of greater 
value than their S.D.I, service and a further 25% considered that it would have 
the same value. 

h) The Information Centres 

The ei£^t inforoiation centres e3q;>ressed a qualified interest in the 
service. Most centres appreciated that the service could provide suppleoentaxy 
material, usually amounting to approximately 10% of their total input. Their 
interest in subscribing to a permanent service depended on their assessment 
of the value of this additional material. They did not expect that the 
service would result in a significant saving of the time spent in manual 
searching as it would still be necessazy for them to search the current 
jotiznals in order to identify references as soon as possible. 

5. ]!'UTUhE VIABILITY OF A |A PREVIEWS SERVICE 

The experiment has shown that there is a substantial interest among 
biologists in a mechanised S.D.I, service and that BA Previews would be an 
acceptable data base for such a service. It was decided that the level of 
interest exhibited by participants Justified the establishment, as a sequel 
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to the present project, of a cost-recovexy experiment at the University of 
Nottln^am. This decision has been vindicated by the fact that ^Sk 
participants in the earlier project were willing to pay a £10 subscription, 
in addition to providing feedback, for their profile during the first year 
of the new experiment.* However, this charge is substantially below the full 
cost-recoveiy price of a B4 Previews service which would probably be in the 
range £50-£75. 

The extent of the demand for a permanent cost-recoveiy service and the 
viability of such a service will be determined by: 

(i) the level of interest in the service among biologists 

(ii) the willingness of organisations to make available the necessaiy 
funds 

(iii) \diether or not large industrial or government-funded research 
Institutions elect to operate their own internal information services 
(possibly by purchasing the BA Previews tapes) rather than purchasing 
profiles from an outside organisation 

The results obtained in the present project do not allow meaningful 
predictions as to the proportion of research biologists who would be willing 
and able to purchase a cost-recovery service during the next five years. In 
partic\2lar it must be noted that: 

(i) the experimental popiaation was biased in at least one respect: 
participants were likely to be enthusiastic consumers of information 
with an above-average interest in the development of new services 

(ii) the reactions to the service, described in this report, are those 
of the Individual research woricer, who is unlikely to be responsible for 
the allocation of funds for these services. In future experiments, it may 
be desirable to assess the reactions of those who will have this 
responsibility. 



Enquiries about this new project should be addressed to Mrs J Stow, 
Department of Botany, University of Nottingham. 
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GLOSSARY OF TERMS USED IN THE EXPERIMEKT 

1. BioSystematic code : a 5-digit taxonomic code 

2. CROSS codex a 5-digit subject code, which may be assigned at one of 
three levels. (See Primary, Secondary and Tertiary codes) 

3. CRSj the prefix used to denote a CROSS code on the KDP-9 tape records 

h» Enrichment term : keyword necessary to clarify or amplify the title - 
added at the end of the title by the BIOSIS editors 

5« Evaluee: participant in the experiment who took part in the quantitative 
evaluation phase (where there is no possibility of misunderstanding, 
evfiduees are simply referred to as 'users*) 

6, Feedback card : a card accompanying each computer printout on which 
the user was asked to mark the number of relevant references and 
%frite brief comments on the service, where applicable 

7. Group profile : a profile which was written to satisfy the information 
needs of two or more workers, who attended the interview and contributed 
to the design of the profile. 

8« IMPORTANT reference : "a reference directly related to your research 

interests, which you will read as soon as possible, i.e. it is essential 
that you know of it." (See MAJOR VALUE) 

9. M^: the prefix used to denote a primary CROSS code on the KDP-9 tape 
records 

10. MAJOR VALUE reference : "very centrally related to your research 
interests and could make a significant contribution." 

11. MINOR VALUE reference : "of interest, and happy to have been informed 
of it, but of no great loss to have missed it." 

12. NKC: the prefix used on the KDP-9 tape records to denote the 
BioSystematic code for an organism which belongs to a new or rearranged 
taxonomic group 

13* Primary CROSS code : the CROSS code which corresponds to the printing 
assignment of the article, i.e. the subject heading under which it 
appears in the printed Biologicsd Abstracts 

1i|. RELEVANT reference : "a reference which, though not important, is of 
some interest to you, i.e. you are glad that you were notified of it." 
ERJC (See MINOR VALUE) 
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15. S»D,I. f Selective Di sBemination of Inf oimation l t a pereonaliBed 

service, whereby individuals or groups are notified of the existence 
of current literature of potential interest. Their interests are 
described by means of a user profile. 

Secondary CROSS code; a CROSS code used to indicate an important 
topic mentioned in the article, other than the printing assignment 
(Primaiy) code. (There may be one or more allocated to each article) 

17. Shared profile t a profile which was written for one person, acting 
on behalf of one or more other workers 

18. SUB: the prefix used to denote a secondary CROSS code on the KDP-9 
tape records 

19. Sira: the prefix used to denote a BioSystematic code on the KDP-9 
tape records 

20» Tertiary CROSS code: a CROSS code used to indicate a topic mentioned 
in the article which is of marginal interest. (There may be one or 
more cLLlocated to each article) 

21. User: participant in the experiment, who may or may not have taken 
part in the quantitative evaluation phase (includes all evaluees) 

22. User profile (or Se arch profile d, a collection of terms (i.e. words, 
codes, journal titles, authors* names) and of logical relationships 
among those terms, which define a user's interests in the form 
required for matching against a machine-readable data base 
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TIME-TABLE FOR THE EXPERIMENT 



Date 


Operation 


Place 


No. of Users 




1. DeveloDment Dhase 






1969 










rit Hxxxxciiacs uOUK up UUUXOB 1 

1 


uxi or Q 




1970 


• 






I January 


J. Stow took up duties 


Nottingham 




9 January 


Seminar: recruitment fcr pilot 
Study (M. Williams) 


Oxford 


16 


26 January 


B. Smith took up duties 


Oxford 




26-30 January 


N.L.L. Course for Information 
Officers in Univeraitv 
Libraries and others 






j 4 February 


Seminar: recruitment for pilot 
study (j. Stow) 


Nottingham 


28 


March 

I 


Completion of pilot studies 


(Oxford 
(Nottingham 


27 
28 


' April 

1 
t 


Visits to INSPBC N T. T. ft+r> 




xTOjeco ooazi 


j May 1970 - 
April 1972 


Studies of data base:- 

1. CROSS and BioSystematic codes 

2 . Enrichment terms 


Oxford 
' Nottingham 


B. Smith 
J, Stow 


May - 
August 


Compilation of "Alphabetical 
index to CROSS Codes" 




Pro;ject Staff 


12 May 


Advisory Committee 


1 

Oxford 




July— August 


Yiait to BIOSIS in the Tmited 
States 

2. ODerational phase 


U.S.A. 


H. Williams 


September- 
December 


Main recruitment of users 




353 


26 November 


Advisory Committee 


Oxford 





ERIC 



- 120 - 



TIMB-TiBLE FOR THE EXPERIMEKT (C0NT»L) 



Sate 


Operation 


Place 


No. of Users 


1971 








Februazy-Aprll 


Second Intezvlews 


— 


353 


Febxuazy 


Reminder letter to users failing 
to return feedback cards 




11 Deleted 


i^rll 


Comnencement of Evaluation 






7 May 


Letter asking users if they 
would take part in Evaluation 




31*2 


18 May 


Idvlsoxy Committee 


Nottingham 




0-15 J'une 


Distribution of BV-6 foxms 
asking for references found 
by searching current literature 
(6-week period) 


- 


151 
(Eraluees) 


22 June 


Last date for profile amendments 


- 




28 JiDie 


Letter to users whose profile had 
been deleted, with questionnaire 




59 


July-August 


Vriting of Medlars profiles (BS) 
Writing of CAC profiles ( JS) 
Heaiffo. of Final Questionnaire (MV) 


Oxford 

Nottingham 

Oxford 


52 
50 


16 Au^t 


Final questionnaire (draft) 
Pilot study distribution 


Oxford 


55 


1,9,15,28 
September 


Precision feedback: distribution 
of outputs representing the 
precision samples 


Nottin^^iam 


151 


23 September 


Final questionnaire: distribution 




302 


October 


Submission of SCI searches to ISI 


Oxford 


Sh 


October- 
November 


Final visits to users 




270 


12 October 


Medlars printouts: distribution 


Oxford 


52 


18 October 


CAC printouts: distribution 


Nottin^am 


50 


15 November 


Document samples: distribution 


Oxford 


151 
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TIME-TiBlJ! FOR THE EXPEHIMEKT (CONT«I)) 



Date 


Operation 


Place 


No. of Users 


19 November 


AdvlBoxy Committee 


Oxford 




1 December 


Output from SCI search; distribution 


Oxford 




1222 








Januaxy - 
May 


inalysis and editing of feedback. 
Calculation of perfoxmance 
pa:raffleter8 






March-June 


Beport writing 


Oxford 




30 June 


Submission of final draft of 
report to OSTI and members of 
advisor'^ committee 






21 July 


Advlsozy Committee 


London 
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B IBLIOGRAPHY 

[Note: citations 1 > 7 are all published by BIOSIS (Philadelphia, USA). 
With the possible exception of reference k (no definite information available) 
all these publications are revised annually. The 1970-2 editions were used 
in the present experiment, ill the BIOSIS publications are available to 
subscribers to BIOSIS services with the exception of references 3 and k]» 

1. BIOSIw A Guide to the Content of BA Previews. 

[Contents include a description of the tape records, and details of the 
categories of enrichment terms used by BIOSIS, of the abbreviations used, 
and of additions, deletions or re-definitions of CROSS or Bio'^stematic 
codes]. 

2. BIOSIS. CROSS Code. 

[a list in code niamber order of about 550 5-digit subject codes]. 

3? BIOSIS. Section Definitions. 

[An indexer's manual which gives detailed definitions of the subject 
areas covered by each CROSS code]. 

h* BIOSIS. List of Priorities (1969). 

[instructions for the allocation of a primary CROSS code in cases 

where there are two or more alternatives, i.e. two or more subject areas 

are equally important to the main theme of the paper]. 

5* BIOSIS. BioSystouatic Code. 

[a listing of about 750 5-digit taxonomic codes, 'each of which represents 
a subdivision of the animal or plant kingdoms, according to phylum, class 
or group of relatad genera]. 

6. BIOSIS. Profiling Guide fcr BA Previews . 

[instruotions for the construction and amendment of BA Previews profiles. 
Issued for the guidance of subscribers to the C.L.A.S.S. sei*vice offered 
by BIOSIS.] 

7. BIOSIS. List of Serials. 

[List of serial publications which are scanned by BIOSIS]. 

8. F.¥. Lancaster. MEDLARS t Report on the Evaluation of its Operating 
Efficiency. Amer. D20. 20, 119-11+2 (April, I969) and also: U^S. Govt . 
Res . Rept . PB I78 660. 



In-house publications not available to tape subscribers. 
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9. P. Leggate. The Evaluation of Operational Current Awareness Services: 
a Discussion of Practical aiid Theo.vetical Problems of Experimental 
Design. OSTI Report No. 5111. Experimental Information Unit, Oxford, 
1971. 

10. P. Leggate, B.N. Rossiter, J.P.B. Rowland. The Evaluation of an SDI 
Service Based on the Index Chemicus Reffistry System . J . Chem . Doc. , 
November, 1973 (in press). 

1 1 . UKCIS. Search Manual ( 1 971 ) . 

12. Experiment with BA Previews . OSTI Newsletter . December I969. 

13. Literature Taped. Nature , 225 . 989-99O (1970). 

II+. P. Wilcoxon. Individual Comparisons by Ranking Methods. Biometrics 
Bull., 1,, 80-83 (December, 191+5); and also Probability Tables for 
Individual Comparisons by Ranking Methods. Biometrics . j[, 119-122 
(September, 191+7). 

15. S. Siegel. Nonparametrr.c Statistics for the Behavioral Sciences. 
McGraw Hill Book Company Inc. 1956. 

16. Westat Research, Inc. Procedural Guide for the Evaluation of Document 
Retrieval Systems. U.S . Govt . Res . Rept . PB 132 711 (December I968), 
PP 137-185. 

17. A. Kent, J. Belzer, M. Kurfeerst, E.D. Dym, D.L. Shirley, A. Bose. 
Relevance Predictability in Information Retrieval Systems. Methods 

• Inf. Medicine . 6 (2), ii6-5l (1967). 

18. M.G. Corfield, R.J. Firth, G. Praser, D. Hartley, P. Leggate, M.M. Korgett, 
J. Riley, B.N. Rossiter. The Lizdson Scientist Experiment: A Study of 
the Provision of Mechanised Services to University Chemists. OSTI 
Raport No. 51^9, Experimental Information Unit, Oxford, May 1973* 
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APPENDIX II 
CONTENTS 

1. STUDY OP THE EDITORIAL PROCEDURES PGR ADDING KEYWORDS TO 
AUGMENT TITLES IN BA PREVIEWS 

2. STUDY OP BIOSYSTEMATIC CODE USAGE 

3. SPECIMEN PROPILE AND PRINTOUT PROM BA PREVIEWS 

U. EXPLANATION OP BIOLOGICAL ABSTRACTS OUTPUT (1970) 
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APPENDIX II 1 . 

STUDY OF THE EDITORIAL PROCEDURES FOR ADDING 
KEYWORDS TO AUGMENT TITLES IN BA PREVIBV/S — 

CONTENTS 

A. INTRODUCTION 

B. AIMS 

C. IMPORTANCE OF ENRICHMENT TERMS IN SEARCHING BA PREVIEWS 

D. METHODOLOGY 

E. RESULTS 

1. Physical descriptors 

2. Organism(s): specific name, common name 

3. Geographical locations 
U. Drugs 

5. Drug affiliations 

6. Specific InstiTumentatlon 



A. INTRODUCTION 

As the BA Previews printout contains no abstract, the user must Judge 
the relevance of the source document. In terms of his existing Interests, 
from the title only. 

In an attempt to make the title more meaningful. It Is the accepted 
policy of BIOSIS to add keywords. No set thesaurus of terms has been 
complied, so that this process Involves the selection of natural-language 
terms by the editors. However, BIOSIS^ ''Ogives some Information concerning 
the categories of keywords which they Include In their title enrichment 
process: "Keywords necessary to clarify or amplify the title are added at 
the end of the title In the following approximate order 

- physical descriptor 

- organlsm(s), specific name, common name 

- organ system, or tissue used or affected 
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- geographical location, Including geological strata 

- drug(s) and afflllatlon(s) plus the word DRUG(S) linked to the 
name of a drug by a hyphen 

* Importauit chemicals 

- specific chemical descriptor (affiliation) 

- specific enzyme(s) 

* specific Instrumentation, apparatus, or methodology (when new, 
the word NEW Is Inserted ahead of the term It modifies) 

- specific diseases 

- purpose of experimentation" 

B. AIMS 

The purpose of this study was to examine some of these categories and 
determine the extent to which torms are added by the editors* From the 
point of view of profile construction It was considered Important to 
determine the consistency with which certain categories of terms are added 
to ascertain if there was any con'wrol of synonyms* On the basis, of the 
findings, it was hoped that some conclusions could be reached as to the 
degree of confidence which could be placed upon a significant term {e.g. 
hormone, drug, specific name of organism) not present in the title being 
added as an enrichment* 

For example, a user who wishes to be alerted to references concerning 
the drug "Bethanidine'' can only hope for 100% retrieval efficiency if he 
can rely upon "Bethanldine" always being added as an enrichment, if it 
is not present in tho author title* 

C* IMPORTANCE OF ENRICHMENT TERMS IN SEARCHING BA PREVIEWS 

In a random sample of 100 references, it was found that an average of 
3*1 words or phrases were added to the author title* 

A sample of l50 references, which had been retrieved by word terms 
only, wae selected from the output of five profiles* It was found that : 

56% of the articles were retrieved by title words only 

31% enrichment words only 

13% " either title or enrichment 

words 
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Enrichment teims are therefore responsible for a substantial 
proportion of the references retrieved. 

D. MEmODOLOGY 

In this study, six categories of enrichment terms were studied j 
Physical descriptors; a list of physical descriptors was compiled 
by scanning (a) the BASIC index of Biological Abstracts and enriched 
titles of BioResearch Index and (b) a computer-produced frequency list 
of enrichment terms used in six issues of Biological Abstracts and 
BioResearch Index . 

Organisms, Geograph ical location and Drug namea ; these were invest- 
igated by scanning the abstracts inappropriate subject sections of the 
printed Biological Abstracts for the presence of these categories of terms; 
their addition as enrichment was then checked in the BASIC index. Checks 
were also made for the additior of geographical locations to titles in 
BioResearch Index . 

Drug affiliations a nd Specific instrumentation : these were invest- 
igated using output from experimental profiles run against several issues 
of BA Previews (both Biological Abstracts and BioResearch Index ^. 

E. RESULTS 

• 1. Physical descriptors 

The following terms which are used to describe publication types 
were found added as enrichment: review, abstract, letter, symposium, 
book, lecture, note, report, thesis, conference. 

The descriptors "symposium" and "book" are added only when the 
complete publication is cited, and not to citations of individual papers 
or chapters (Pig. 1). 

2. Organism(s)i spe cific name, common name 

BIOSIS^^ ^states that specific and common names are added "10 maximua 
in a series; if more are mentioned the individual taxa are grouped in 
larger categories". Table A shows samples of this type of enrichment. 

It was found that the specific and common names of organisms are 
consistently added when they are not mentioned in the title but are 
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referred to in the abetrfiujt (Tebles B and C). If an article mentions the 
common name in the title and the specific name in the abstract, the latter 
would be included as an enrichment term (Table D). Conversely, if the 
specific name is mentioned in the title and the common name is in the 
abstract, it was found that the common name is not added as an enrichment term 
(Table E). If both the common name and specific name are referred to in 
the abstract, only the specific name, in general, is added as an enrichment 
term (Table F)« 

It is difficult to restrict a search to "humans" by using natural- 
language terms, as "human" could appear in the enriched title in numerous 
forms. The following have been observed: hitman, girl(s), boy(s), 
adolescent (s), adolescence, man, men, woman, women, infant, child, childhood, 
children, brother(s), patient, subject. For restricting a search to "human", 
the BioSystematic code for Hominidae (SYS86215) must be used. 



Figure 1 . ADDITION OF THE PHYSICAL DESCRIPTOR "STOPOSHM" 



COSTAMTINO I.N. 

PROCEEDINGS OF THE LATIN AMERICAN CONFERENCE 
ON THE CONSERVATION OF RENEWABLE NATURAL 
RESOURCES SAN-CARLOS-DE-BARILOCHE ARGENTINA 
MARCH 27 APRIL 2 I968 = SYMPOSIDM 
INT UNION CONSBHY NATURE NATUR RES ANNU REP 
(13). 1968,9l4-U63 

NUMBER 15981 

(a) Complete publication: "Symposium" added as enrichment term 



CRESPO J. A. 

VAMPIRE AND OTHER BATS AND THEIR 
CONSERVATION=DESMODUS-ROTUNDUS ECOLOGY 
INT UNION CONSBHY NATORE NATUR RES ANNU REP 
(13). 1968, IO2-IOI4 

NUMBER 1598I4 

(t) Individual paper: "Symposium" not added as enrichment term 



3. Geographical locations 
(1) 

BIOSIS^ 'states: "A recognised geographical area (usually a country) 
is added at the end of the title for any geographical region occurring in 
the title or abstract". 
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a) Location given in the title 

In general, if a specific city, tovm, county, river, mountain, estuaiy, 
lake, forest, research station, zoo, national park etc. is mentioned in a 
title, the corresponding country is added as an enrichment teim. 

Of 250 references which cited a geographical location of some type 
in the title, only 23 did not have the corresponding country inserted 
(Table G). 

b) Location given in the abstract only 

It was found that, when the geographical location is mentioned in the 
abstract but is not present in the title, it is not consistently added as 
an enrichment term. 

Of 100 references which had a geographical location mentioned in the 
abstract, only 21, had the corresponding country added as an enrichment term 
(Table H). It appears that, if the site of the research is mentioned in the 
abstract, but is not considered to be an important feature of the work, the 
title is not enriched with the corresponding country. However, refervances 
were found where the location did not seem to be incidental to the research 
(Table I). 

c) Conclusion 

At present, it must be accepted that a profile cannot be effectively 
restricted to any particular geographical area, as it was not possible to 
find any standard procedure for this type of enrichment. Even if a location 
is mentioned in the abstract, it may not bu added as an enrichment. 
Purtheimore, there appears to be Inconsistency regarding the choice of 
enrichment, e.g. references relating to a location in Great Britain could 
be enriched with Great Britain, Britain, British Isles, United Kingdom, 
aigland, Scotland or Vales. The form used tends to be that given by the 
author. 

|.. Drugs 

BIOSIS^ states that drugs are added "10 maximum in a series; if mora 
are mentioned all drugs are grouped in larger categories". Table J shows 
examples which indicate that this statement is not always adhered to. 

A sample of 77 references was examined of which 73 had all the drugs 
mentioned in the abstract inserted as enrichment terms. In a few cases 
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drugs not mentioned in the abstract appeared as enrichment tenns and must 
therefore have been mentioned in the original article. 

There appears to be no consistent policy regarding the addition of 
drug compounds when trade names (or alternative names) are 'given in 
brackets. Table K shows that in some oases both are added; in some the 
f oimer and in others the latter. BIOSIS^ states that generic names are 
preferred to registered names. 

$» Drug affiliationfa ^ 

The policy regarding drug affiliations^^ ^ is stated as follows: 

1 . For any drug name appearing in the title, the drug affiliation 
plus the hyphenated word DRUG are added once at the end of the title, 
iny drug name in the abstract is added at the end of the title and treated 
coiaparably. Generic names are preferred to registered names. 
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2. When a drug is discussed as having two different modes of action, 
each action is added. 

3. The affiliations which are used relate to the specific action of 
the drug(s) as described or studied in the paper and not to the usual 
action associated with that drug, 

h* The affiliative abbroviations listed below are based upon specific 
action categories for -r. ii.In.: drugs and conform to the drug classification 
of the United States Pharniacopoia, Seventeenth Revision (l9b5). 

Anesthetic 
antidote 
antihistamine 



antiinfective 

antiinflammatory agent 

antinauseant 

antineoplastic 

antiparasitic 

antiviral 

autonomic drug 

carbonic anhydrase 

cardiovascular 

central depressant 

central stimulant 

dermatological 

diagnostic 

expectorant 

gas t ro i n t e s t i nsLl 

hematologic 

hormone 

immunologic 

metabolic 

uiigraine (specific) 
■jxytocic 

pharmaceutical adjunct 

radioprotectorant 

relaxant 



renal acting 



ANESTHETIC-DRUG 
ANTIDOTE-DRUG 
ANTI HIST-DRUG 



ANTI INPECT-DRUG 
ANTI imAM-DRUG 
ANTI NAUSEi\NT-DRUG 
ANTI NEOPLASTIC-DRUG 
ANTI PARASIT-DRUG 
ANTI VIRAL;.DRUG 
AUT0N0MZ6-DRUG 

CARBQjnC ANHYDRASE INHIB-DRUG 

camIio VASC-DRUG 

CENT-DEPRESS-DRUG 

CENT-STIM-DRUG 

DERMATOL-DRUG 

DIAGNOS-DRUG 

EXPECTORANT-DRUG 

CASTRO INTEST-DRUG 

HEI4AT0L-DRUG 

HORMONE- DRUG 

IMMUNOL-DRUG 

METAB-DRUG 

MIGRAIITE-3PECIFIC-DRUG 

OrrTOCIC-DRUG 

PHARI^-ADJINCT-DRUG 

RADIOPROTECT-DRUG 

RELAXANT-DRUG 

REITAL-ACTING-DRUG 
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The usage of the affiliations "HORMONE-DRUG" and "ANTI VIRAL DRUG" was 
investigated. 

HORMONE-DRUG 

A study was carried out to determine the reliability with which the 
drug affiliation "HORMONE-DRUG" is used to relate to the specific action 
of the drug(s). It was found that "HORMONE-DRUG" was consistently added 
if a particular hormone was used as a drug to observe its effect on a 
particular system. 

ANTI VIRAL-DHUG 

An experimental profile consisting of the CROSS code for anti-viral 
agents (CRS38506) and the term "ANTI VIR*" was run against five issues of 
BA Previews to determine the effectiveness of the drug affiliation "ANTI 
VIRAL-DRUG" compared with the CROSS code. The term "ANTI VIR*" was used 
in order to retrieve natural-language terms for anti-viral agents in the 
author title. 

The results are shown in Table L. Whenever the drug affiliation 
"ANTI VIRAL-DHUG" was added as an enrichment term the CROSS code for anti- 
viral agents was also allocated. The references which were retrieved by 
"ANTI VIR*" and not by the CROSS code are shown in Table M. As articles 
can be retrieved by means of title words only, and not by the CROSS code 
or enrichment, it is not possible to rely exclusively upon the search 
term "ANTI VIRAL-DRUG" as a keyword in the profile. 

It was also noted that an affiliation could be represented by several 
slightly different enrichment phrases, e.g. 

ANTI VIRAL-DRUGS 52 (05)No. 21+38 1 

ANTI VIRAL DRUGS 52(06)No.32257 
ANTI VIRAL-DRUG 52{06)No. 32220 

ANTI VIRAL 52(06)No.31389 

Some references would also be missed if the affiliation alone were 
relied upon, as an agent effective against a virus might be considered 
to be primarily an IMMUNOL-DRUG Lil(02)No. 138U3 ', an ANTI NEOPLASTIC-DRUG 
[2l(02)No.13826J or an ANTI INFECT-DRUG Lll(02)No. I38O6 ]. In such cases 
the enrichment ANTI VIRAL-DRUG would not be inserted and the references 
would only be retrieved by the CROSS code for anti-viral agents. 

The CROSS code for anti-viral agents has a much broader coverage than 



"ANTI VIR*" and retrieves general articles vAiioh are obviously relevant 
to the field, e.g. 

"UMEZAVA, H. Proceedings of the 6th International 
congress of chemotherapy. Section on chemotherapy 
of viral diseases [71 (02)No« 13822]." 

It is apparent that, for complete coverage of anti viral agents, both 
the CROSS code and the natural-language term "iNTI VIR*" must be used. 

6. SDecific Instrumentation 

In order to investigate the addition of instrumentation terns, a 
profile was constructed incorporating natural-language terms for "electron 
microscopy" and the CROSS code for "electron microscopy" (CRS01058). 

The results shown in Table N indicate that the term "electron microBOopy" 
is frequently added as enrichment \rtien it occurs in the abstract. However, 
the grammatical form of the enrichment may differ from that used in the 
abstract. 

Table P shows that, in order to retrieve references concerning 
"electron microscope studies", the enrichment term "electron microscope" alone 
is not sufficient. Enrichment terms used also include "electron micrograph" 
and "electron beam microscopy". The "electron microscopy" CROSS code 
(CRS 01058) especially at MAJ and SUB level, was usually assigned only to 
articles in which the technique itself was being studied and not to articles 
in which only the results obtained by the technique were of interest. 
For this type of profile all the synonyms for "electron microscopy" must 
be included in addition to the CROSS code for micro and ultra-microacoplc 
anatomy (CRS 11108) when appropriate. 
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TABLES 

A. Organisms: Examples of enrichment ^Aien more than 10 species 
are listed in the abstract. 

B. Organisms: Examples of enrichment with common names of organisms 
mentioned in the abstract. 

C. Organisms: Examples of enrichment with specific names mentioned 
in the abstract. 

D. Organisms: Examples of enrichment when the common name is 
mentioned in the title and the specific name in th3 abstract. 

E. Organisms: Examples of enrichment when both the common name 
and the specific name are mentioned in the abstract. 

P. Organisms: Examples of enrichment when the specific name is 
mentioned in the title and the common name in the abstract. 

G. Geogrs^hical location in the title: sample from 250 references. 

H. Geographical location in the abstract: sample from 100 references. 

I. Geographical location in the abstract: subject coverage of 
references. 

J. Drugs: enrichment with drug names when more than 10 drugs are 
listed in the abstraxst. 

K. Drugs: enrichment with drug names when trade names or 
alternative names are given. 

L. Drug affiliations: comparison of retrieval by CROSS code 
(CRS3d506) and natural-language term (ANTI VIR*). 

M. Drug affiliations: retrieval by ANTI VIR* in the title and 
not by the affiliation ANTI-VIRAL DRUG or CROSS CODE (CRS38506). 

N. Specific Instrumentation: relationship between terms mentioned 
in the abstract and teims added as enrichment. 

P. Specific Instrumentation: comparison of retrieval by CROSS Code 
(CRS01058) and by natur^a-language terms. 
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TABLE A. ORGANISMS: EXAMPLES OP ENRICHMENT WHEN MORE 
TSAN 10 SPECIES ARE LISTED IN THE ABSTRACT 



BA VOL 51 

ABSTRACT NO. ABSTRACT ENRICHMENT 



28362 



23933 

3U71U 

3U695 
UOU73 



Boletus edulis, Boletus 
rufus, Boletus scaber, 
Lax5tarius rufus, L 
deliciosus, L tonninosus, 
L rolemus, Birch musn- 
rooms, orange mushrooms, 
fiispen mushrooms, edible 
boletus, pepper mushrooms, 
venain, pine forest, 
fir groves 

quotes 7 individual 
species plus 5 Poly- 
gonum species 

approximately ^0 species 
listed 

quotes 1U Culicoides 

species of TInionidae 
and Margaritif eridae 
listed 



Boletus spp. Lactarius 
spp. Pine, fir, vermin 



7 species 
Polygonum-spp 



1U new record species 

Amblemella new genus 

Benminaia new genus 

Hemisolasma 

1:"'arreysia 

Lamellidens 

limnoscapha 

Nannonala 

Traps zoideus 

Rectidens, Physunio 

Unio Pseudodon 

Monodonlina 

Leguminaia 

Cosmopseudon 

28 new spacies 

2 new forms fossil 
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TABLE C. ORGANISMS: EXAMPLES OF ENRICHMENT WITH SPECIFIC 
N AMES MENTIONED IN THE ABSTRACT (SAMPLE FROM 1|2 
REFERENCES) 



BA VOL 51 
ABSTRACT NO. 


ABSTRACT 


ENRICHMENT 


35110 


Hlppurls vulgaxis 


Hippuris- vulgaris 


35169 


C« bliiinely C« barabatus 


Coleus-»blumei 
Coleus'-barabatus 


35180 


Glycine max 


Glycine-max 


29729 


Plantago lanceolata 
Rumex acetosa 
Dactyl is glomerata 


Plantago-lanceolata 
Rumex-acetosa 
Dactyl i s<-glomerata 


297liO 


Cxyptostegla grandlflora 
Fergularla daemla 
Calotrjpls procera 
Leptadenia pyrotechnlca 


Cxyptostegia^grandiflora 
Fergul ari a-daemia 
Calotropis-procera 
Lep tadenia-pyro te chni ca 


29750 


Thalaapl alpestre 
Hinufitrtla veina 


Thai aspi-- alpestre 
Hinuartia-vema 


29756 


Deroceras retlculatum 
Arlon fasclatus 
A* subfuscus 
A« Intexmedlus 

D# caruanae 


DeTOceras^-reticulatum 
Arion--fa8ciatus 
Ari on» sub f us cus 
Arion-intermedius 

DeroceraS'-caruanae 


3li550 


Camerlna-catenula 
Eoconuloldes parvulus 
Eoconuloldes wellsl 
Hellcosteglna polyfyralls 
Hellcolopldlna spiralis 


Camerlna- catenuia 

Eo c onul 0 i de s-paxvul us 

Eoconuloides-vellal 

Helicosteginar-polygyralis 

Helicolepidinap>spiralis 


33622 


Biomphalaria tenagophilia 
Schistosoma mansoni 


Biomphalaria^ tenagophila 
Schistosoma-mansoni 


23923 


Pistacia lentiscus 
Quercus ilex 


Pistaciar^lentiscus 
Quercus-ilex 
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TABLE E. ORGANISMS » EXAMPLES OP ENRICHMENT WHEN BOTH 
THE COMMON NAME AND THE SPECIFIC NAME ARE 
MENTIONED IN THE ABSTRACT 



BA VOL 51 
ABSTRACT NO. 


i 

AOwXXUivX 


iiirixllOnfUliNT 


2hp26 


Field vole, Microtus arvalis 


Microtus-arvalls 




Fleas 9 Amphallus«*runatus 
Ctenophyllus bondarl 


Amphallus*runatus 
Ctenophyllus-bondari 


U7156 . 


Great homed owl 

Bubo vlrgii lanu%Long eared 

0Wl« AslO^-otUB « BUTTPO Vl TiSt 

owl 9 Speotyto cxmiciLlarla 
Barn owl 9 ^to alba 


Bubo-vlrginlanus 
Aslo-otus 

O^to-alba 


b28l1 


Lettuce 9 Laotuca satlva 


Lac tucar- satlva 


35022 


Sweet chestnut 
Castanea satlva 


Caetanea^satlva 


i|1273 


FagU89 Magnolla9 Quercus9 
Caxya9 Llquldambar9 
Pinus9 Illicium floridanum 
Beech9 magnolla9 oak9 
hickory9 sweetguin9 pine 


Fagus9 Magnolla9 ^ercus9 
Carya9 Llquldambar9 
Plnus 

IlllclunH floridanum 


U1327 


Yellowfin tuna 
Thunnus albacares 
Blgeve tuna 
T« obesus 


Thunnus* alb ao are a 
Thuxmus-obesus 


3802U 


Kidney beans 
Phaseolus vulgaris 


Fhaseolus-vulgarls 


29789 


Salmo trutta 

Brown trout 9 Brook trout, 
Salvellnus fontinalls 


Salmo- trutta 
Salvellnus- fontinalls 


29802 


Bay mussel 9 llytllus edulis 
Oligoxocottus synderi 


Mytllus-ediills 

\j± 15OXOCOXXUS— synueri 


3ii355 


Chestnut blight 
Endothla paxasltlca 


Endothla-parasltlca 


3U876 


Blowfly, Calllphora phae- 
nlcla 

Housefly, Musca domestlca 


Calllphora^phaenlcla 
Mu oar^domestloa 


23298 


Olive fly, Bacus oleae 


Olive flyf Dacus-oleae 


18070 


Coho salmon, 
Oncorphynchus kisutch 


Qncoiphynchus-klsutoh 


12150 


Lololly Fine, Plnus-taeda 


Plnus-taeda 


17390 


Crayfish, Panulirus longipes 


Panul Irus- longlpe s 


5822U 


Skipjack tima, yellowfin 
tuna, bluefln tiina, 
Katsuwonus pelamls 
Thunnus albacares 
Thunnus thynnus 


Katsuwonus-pelamls 
Thunnus-albacaros 
Thtumua- thynnus 



TABLE p. ORGANISMS? EXAMPLES OP ENRICHMEOT WHEN THE 
SPECIFIC NAME IS MENTIOIJED IN THE TITLE AND 
THE COMMON NAME IN THE ABSTRACT 



BA VOL 51 ENRICH- 
ABSTRACT NO. TITLE ABSTRACT MENT 





lie uc;xpugun con uoruus 
Themeda australls 


black spear grass 
kangaroo grass 


1;1261 


Qjxercus turbinella 


live oak 
emory oak 


1;1305 


Shaerlum come\ini 


fingernail clam 
uoaa 


29720 


npfiym+.ViiiQ vpI 11 + 1 nil Q 


snowuusii 




T^l^ T 0 0 1 11 1 W Q 0 T A VI ai 1 tf* 

IxX UUXXtUU CaS w6n6Ulu 

Trlbolium confusum 


recLrusu iiour oeetie 
confused flour beetle 


19539 


Trltusus pyrrhogaster 


Japanese newt 


22906 


Llssocarclnus 
ecnxnocxisci 


crab 


1;099U 


Arb^mla salina 


brine shrimp 


3U815 


Pollnlces lewisi 


clam drill 


31+990 


Eutamlas 

(JuoaIPx V X u u a UUS 


chlpmiinks 




no pue um vui gape 


barley 


3U078 


Glycine max 


soy beans 


31+091 


Lemna glbba 


duckweed 


39701 


Theobroma cacao 


cacao tree 


390I2 


Allium cepa 


onion 


39962 


Pinus e dulls 
Pinus alblcaulls 


piny on pine 
whltebank pine 


22229 


Pi sum sativum 


peas 


221+22 


Cynodon dactyl on 


Bermuda ^^rass 


22U29 


Elymus junceus 


wild rye 


226JU 


Plutella maculipennis 


diamond black moth 
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TABLE H. GEOGRAPHICAL LOCATION IN THE ABSTEACIR 
SAMPLE PROM 100 REFERENCES 



BA VOL 51 
ABSTRACT NO. 


ABSTRACT 


ENRICHMENT 


23952 


DerDysnire 




35520 


Oa««^V« ^ P j-i -umi \ A 

bOUtn \jB±HOVtllBL 


^axiiomia uda 


1001 ). 


Southern Sweden 




18117 


Africa^ Uganda^ Kenya 
South Africa 




IoIdO 


Pisa Florence 




355OO 


ooloraao ciesert 




35510 


Wisconsin 


- 


35583 


Marion Lake 


- 


18021 


Africa^ Australia^ America 


- 


!.0230 


England) Scotlandi Wales, 
oarpaxnxan provenances 


United Kingdom 
u zecnoB i. ovcuu.ei 




ne rgao xu. ve r v aii ey 


UwwA 


727 


tl^l 1 TTCOT3 

iioii and f Ubbi(| 

Africa, S* Europe 




35536 


Nova Scotia, New Brunswick 




58751 


Oregon coast 




61.655 


Georgia 




6),657 


JventucKy 




61.665 


Oklahoma 


- 


61.668 


Nevada 


- 


61.670 


Michigan 


- 


61.671 


Newport River Estuary 
S* Caroline 




61.678 


Texas, Mexico 




1.7137 


W» Hungary, E» Poland, 
N & S German Hill Country 




1.7162 


Toronto 




1-7169 


Swedish Lake 




1.717t- 


Nanao Bay Japan 


Japan 
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TiBLE J. DBUGSt SNBICBMERT VITH IfflUG NiHES VEEN MODE 
TEAS 10 DRUGS ARE LISTED IN THE ABSTRACT 



BA VOL 51 
ABSTRACT HO, 


ABSTRACT 


; NO 

1 ENRICHMENT 

1 TERM* 


136538 


ohloipromazine, 
trifluoperazine , 
pexphenazlne, thioridazine, 
haloperidoli ethomoxane, 
chloxpro thixene , 
tetrabenazine, resezpine, 
benztropine, 
chlordiazepoxide , 
pentobarbital, 

meprobamate, 1 
chloral hydrate, 
paraldehyde, scopolamine, 
cyproheptradine, atropine, 
trihe:gnpl^0i^dyl 




37856 


phenelzine, iproniazid, 1 
iso carboxazide, 1 
iffiipraaiine, hazmaline, 
pheniprazine, pargyline, 
t ranyl cypramine , , 
amitriptyline, cocaine, 1 
asiphetaffliiAe, BL-BOPA, ' 
5-hydroxtiyptophan | 


imipramine 
amitriptyline 


49250 


aduL^'oran, librium ! 
mogadon, Valium, 1 
tegretal, tofranil, 1 
pertof ran, insidon, | 
noveril, laroxyl, acetexa | 




U9370 


papaverine, euphyllin, | 
nitroglycerine, chloraolzine, • 
chlorpromazine, reserpine, ' 
hexamethanium, morphine, » 
promedol, fibrinolysin, ' 
isonlcotinic acid, ipiazide | 





In all cases except for the two cooipounds named, the drug 
name appears as an enrichment tezm. 
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TABLE L. DRUG AFFILIATIONS: COMPAfilSON OP HETRIEVAL BY 
CROSS CODE (CRS38506) AND NATURAL-LANGUAGE 
TERM (ANTI VIR*) 





RETRIEVED BY 


RETRIEVED BY 


RETRIEVED BY 


ISSUE 


BOTH ANTI VIR* 


ANTI VIR» 


CRS38506 




AND CRS30506 


ONLY 


ONLY 


71(02) 


1.0 


3 


17 


52(03) 


1. 




3 


52(0).) 


t. 


1 


1 


52(05) 


13 




6 


52(06) 


11 


2 


6 



TABLE M. DRUG AFFILIATIONS: RETRIEVAL BY ANTI VIR* IN THE 
TITLE AND NOT BY THE AFFILIATION ANTI VIRAL-DRUG 
OR CROSS CODE (CRS38506) 



Abstract No. Terms in Title 



52(0).)20139 


anti 


viral 


antibody 


52(06)31635 


anti 


viral 


antibody 


52(06)32216 


anti 


viral 


activity 


71(02)10700 


anti 


viral 


action 


71(02)1003). 


anti 


viral 


resistance 


71(02)13983 


anti 


viral 


substance 
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TABLE N. SPECIFIC INSTRUMENTATION: RELATIONSHIP BETWEEN 
TERMS MENTIONED IN THE ABSTRACT AND TERMS 
ADDED AS ENRICHMENT 



BA VOL 52 
ABSTRACT NO* 


CRSOI058 




TITLE 


ENBICHMENT 


ABSTRACT 




y 


Fine structure 


• 


Electron ' 


128180 


y 








microscope 


tt 


tt 














12821.7 


y 








nil 01^inB^/\T\A 

lux^fvowOpo 


\% 


tt 

• 






127090 










in n ^ A 


tt 


tt 




1261.30 




tt 


tt 




— 


126127 


/ 


tt 


tt 




- 


128173 


J 


If 


II 


Electron 


Electron 


128190 








microscopy 


microscope 


J 


II 


II 


Electron 


Electron 


128252 








microscopy 


microscope 


J 


. II 


II 


Electron 


Axec uron 


127126 








microscope 


microscope 


J 


II 


II 


Electron 


Electron 


12J.071. 








microscopy 


microscopy 


11 


tt 


Electron 


Electron 


123703 


J 






microscope 


microscopic 


II 


II 


Electron 


Electron 










microscope 


microscope 



TABLE P. SPECIFIC INSTRUMENTATION » COMPARISON OP RETRIEVAL BY 
CROSS CODE (CRS01058) AND BY NATURAIr-LANGUASE TERMS 



ISSUE 


CRS01058 
ONLY 


CRSOIO58 
+ ELECTRON 
MICRO* 


CRS01058 
+ ULTRA 
STRUCT* 


CRS01058 
+ PINE 
STRUCT* 


ELECTRON 
MICRO* 
ONLY 


ULTRA 
STRUCT* 
ONLY 


FINE 
STRUCT* 
ONLY 


71(10) 


2 


97 


1 




3 


8 


52 


71(02) 


10 


90 


15 


3 


f. 


5 


22 


71(03) 


2 


115 


2J. 


5 


1 


7 


25 


52t03) 


22 


123 


23 


2 


5 


18 


' 20 
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APPENDIX II 2 . 
STUDY OF BIOSYSTEMATIC CODE USAGE 
CONTENTS 

A. METHODOLOGY 

B. RESULTS 

1. Coding omissions 

2. Coding errors 

3. Studies of BioSystematic code allocation in six 
areas of applied biology 

U. Scope of very general BioSystematic codes 

5. Overlap between CROSS and BioSystematic codes 

C. RECOMMENDATIONS FOR USING BIOSYSTEMATIC CODES 

1. Phylogenetic considerations 

2. Subject range of BioSystematic codes 



A. METHODOLOGY 

Studies of the allocation of BioSystematic codes were carried out, using 
specially constructed profiles which were run agednat one or more issues 
of BA Previews . The output from these experimental profiles included a 
printout of the codes allocated by BIOSIS to each paper. For some aspects 
of the study, it was also necessary to scan tho corresponding abstracts 
in the hard copy Biologiceil Abstracts but, with one exception, reference 
was not made to the original, article. 

B. RESULTS 

1. Coding Omissions 

In order to determine whether BioSystematic codes axe always assigned 
when an organism is mentioned in either the title or the abstract, word 
termc and taxonomic CROSS codes were used to construct profiles correspond- 
ing to seven taxonomic groups. Table A shows the results of an ana?ysis 
of the articles retrieved by each of the seven profiles. 

ERIC 
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The four articles in which an organism was mentioned in the enriched 
title, but the appropriate BioSystcmatic code was not allocated are listed 
below. In 'each case, the organism for which a code was not assigned is 
indicated by underlining. 

"Electron microscopic observations on the association of viruses with 
membrane systems in hamster tumor cells propagated in tissue culture" 
(il, No.;>, Abs.U9992) 

"Attempt to increase the effect of regional chemotherapy in experimental 
tumors by procaine" s virus transformed rate fibroblasts exposed to 
thymine 

(il. No. 9, Abs. 50008) 

"Antibodies to Epstein Barr virus in Burkitt's Lymphoma and control groups" 
(il, No. 9, Ab3.U9c3UO) 

"Malignant and transforming activity of Rous sarcoma virus . Vaxt 1. 
Msdignant effects of Rous sarcoma virus " 
(il, No. 9, Abs.U99(J6) 

However, the appropriate BioSystematic code was allocated in all the 
other i4J47 articles which were examined. It can therefore be concluded that, 
if an organism is mentioned in the enriched title and/or the appropriate 
taxonomic CROSS code is allocated, the appropriate BioSystematic code for 
the organism is assigned almost invariably. 

2. Coding Errors 

One or more BA Previews tapes were searched for all articles containing 
the BioSystematic codcb for five taxonomic groupings. The seMch profiles 
made us^ of the partially hierarchical structure of the BioSystematic codes. 
Thus the search terra SYSO} would retrieve articles which w^.d been assigned 
any of the following codes: 

SYSO3OOO (Viruses - general); SYS03200 (Animal viruses); 
SYSO3UOO (Plant viruses); and SYSO30OO (Bacterial viruses) 

The title and abstract appearing in the printed Biologicsd Abstracts were 
consulted for each of the retrieved articles in order to discover whether 



* Words following the equality symbol (r. ) are enrichment terms added by 
the BIOSIS indexers 
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the BioSystematic code had been assigned appropriately. The results are 
shovoi in Table B. 

In all but one of the articles examined the BioSystematic code corres- 
ponded to an organism mentioned in either the title or the abstract. The 
single exception was: 

"Vineyard sprayers' lung. A new occupational disease" = guinea pigs 
copper sulfate hydrate lime toxicity 
(il, No.9, Abs.i49735) 

The article was assigned the BioSystematic code for Echinodermata in both • 
the magnetic tape and printed indexes, but neither the abstract nor the 
original paper contained any reference to this type of organism. This 
appeared to be an isolated clerical or indexer error. The probability of 
a BioSystematic code for a particular taxonomic group retrieving an article 
unrelated to that group is therefore negligible. 

Table B also shows that the BioSystematic code corresponded to an 
organism mentioned in the title In 91% of the articles examined. The 
BioSystematic index therefore appears to be only slightly mors exhaustive 
than the enriched title. However, the codes would usually be preferred to 
natural-language terms, as with the latter allowance must be made for 
synonyms and alternative grammatical forms. 

3» Studies of BioSystematic code allo cation in Big areas of AT^pH ar^ 
biology 

It would be eacpected that in articles concerned with applied biology 
any organisms mentioned would often be of secondary importance to the main 
theme of the paper. The consistent allocation of BioSystematic codes to 
such articles is obviously an advantage to some applied biologists, but 
it was also found to be a disadvantage for users who were interested in 
specific organisms but not in relation to applied biology. For some users 
in this category, the use of BioSystematic codes retrieved an excessive 
number of irrelevant items. 

A study was therefore made of the allocation of BioSystematic codes 
to articles from six fiylds of applied biology. A sample of articles 
from each field was P^trieved by search profiles containing either CROSS 
codes or word terms as shown in Table C. 
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a) Food Technology 

References concerning fruit, \egetables, cereals and fish were retrieved 
by these three CROSS codes in the context of: 
(i) processing as items of food 

(ii) nutritive qualities 

(ili) chemical composition of food 

(iv) spoilage and contamination. 

Of 111 articles retrieved, only d had no BioSystematic code corresponding 

to the organism acting as the source of food and five of these did not 

mention the type of food inv<?lved. The remaining three articles to which 

the BioSystematic code was not assigned were: 

••Vitamin B-6 components in some meats, fish, dairy products and 
commercial infant formulas" 
(51, No. 12, Abs. 65300) 

"Overweight and dietary habits of adolescents = vegetable fats 
butter cheese meat fiah" 
(il, No.ll^, Ab3.7675l; 

"Lycine enrichment of >^eat in flour evaluation in infants" 
(il, No.11*, Abs. 76733) 

b) Forestry & Forestry Products 

A profile consisting of the CROSS code 53500 retrieved articles relating 
to trees and other plants in the context of: 

(i) forest fire risk 

(ii) regional distribution of various species 

(iii) timber products e.g. wood, sawdust, resin 

(iv) climatic conditions 

(v) afforestation programs 

(vi) soil requirements and nutrition 

(vii) forest pests and 'forest animals. 

Of the 15(3 references retrieved, only 8 had no appropriate BioSystematic 
code and none of these mentioned specific genera or species. 
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c) Plant-derived Drugs 

A title-word search for drugs of plant origin retrieved papers concerned 

with: 

(i) therapeutic medicine 
(ii) tissue culture 

(iii) chemical analysis 

(iv) chemical extraction 

(v) pharmacological experiments on animals. 

All but two of the 37 retrieved articles were indexed with the appropriate 
BioSystematic code for the plant from which the drug derived. 

d) Mycology and Antifungal Agents 

References to fungal species were -«trieved by the CROSS codes in the 
context of: 

(i) disease-earring pathogens 

(ii) diagnosis of fungal infection 

(iii) drug treatment of fungal diseases. 

All but five of the kO references retrieved had a fungal BioSystematic 
code allocated. Of these five only two madie even a slight reference to 
•Vcotic infection" or "antifungal treatment"; the remaining three made 
no reference at all. 

e) Tissue Chilture 

This CROSS code retrieved articles concemeji with: 
(i) irradiation of animal cells 
(ii) cell culture media and methods 
(iii) cells used to culture viruses and bacteria 

(iv) pharmacological response of cells in cro 

(v) metabolism of cells in culture. 

In all but two cases, the article was indexed with the BioSystematic code 
for the animal or plant from which the cells were originally obtained. 
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The two exceptions were; 



"The in vitro transformation by an avian adenovirus"^ chick 
embryo lethal orphan virus 
(ii, No.9, Ab8.U9995) 

••Correlation between immunoreactive growth hormone and prolactin 
activity in human and simian pituitary cell cultures'' = fetus 
pigeon crop sac method 
(il, No.9f Abs*50212) 

f ) Clinloal Medicine 

Articles on human clinical medicine retrieved by these CROSS codes 
included such terms as: 



human(s) 

man/men 

woman/women 

mother(s) 

female (s) 

ohild(ren) 

infant(s/ile) 

adult(s) 

adolescent (s) 

aged 9 veteran(s) 

Indian(s) 

hipp(y/ies) 

patient (s) 

case(s) 

clinical 

addict(s) 



alcohol(ism/ic(s)) 

schi2ophren( ia/ ic( s ) ) 

medico«»legal 

forensic 

paternity 

legislation 

public health 

rehabilitation 

drunkenness 

teenageCrs) 

girl(s) 

physically disabled 
medical education 
bab(y/ies) 



and many other terms > all indicative of some aspect of human medicine. Of 
the 112 references retrieved, only two had not been given the BioSystematio 
code for Homo sapiens # These weret 

"Soope of immunology in science^ research and clinical terms" 
(ill No*9f Abs.l50614i) 

• 

"The biologic effects of UV reuiiation with emphasis on the skin" = 
book symposium erythema pt ytoallergy phyto- toxicity 
(^f No*:^, Abs.U6690) 



. 158 - 



Scope of very general BloSyatemfttir. nr^Ai^a 

Several veiy broad categories are included in the BioSystematic Code 
Index, e.g. 

00500 Orgaiiisms (General) 
01000 Micro-organisms (General) 
11000 Plantae (General) 
33000 Animalia (General) 
31+000 Invertebrata (General) 
85150 Vertebrata (General) 
85700 Mammalia (Genoral) 

The BioSystematic index is not fully hierarchical and a search for one 
of these general codes will not retrieve articles which have been given only 
one Of the more specific codes corresponding to the subdivisions of these 
phylogenetic categories. It is therefore necessa^ to know vAxen and how 
these general codes are assigned. If BW^adheres to the stated editor^ 
ial policy of assigning the most specific of the available codes, the general 
codes should only be used for articles ^ich describe general principles 
and/or fail to mention specific organisms. 21,3 references retrieved by the 
codes forMicro-organisms, Chordata and Vertebrata were classified as shown 
in Table D, from which it will be seen that: 

(i) 186 references did not mention any specific organisms though some 

Of these referred to •microbes' , 'fauna', etc. which were not otherwise 
specified. 

(ii) 29 references dealt with general principles though a few animals 
were mentioned specifically. 

(iii) 28 references did not appear to deal with general topics and referred 
to only one or a few species. The allocation of a more general 
BioSystematic code would appear to be a coding error. 

Since the two categories, Chordata and Vertebrata, could overlap, their use 
was examined in more detail x 

a) Chordata (109 articles) 

Topics discussed in references allocated this code included: 
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(i) theoretical concepts e.g. mathematical models, 

cybernetics, 

philosophical discourses, 
(ii) general biology e.g. conditioning theory, 

general dietcay needs, 
disease epidemiology, 

(iii) general methodology e.g. specimen preparation, 

body fluid preservation, 
histological techniques, 
blood flow measurements, 
chemical levels in blood, 
(iv) general biochemistry e.g. biosynthesis, 

enzyme activity, 
(v) general medicine e.g. diagnostic techniques. 

"b) Vertebrata (U8 articles) 

Topics discussed in references allocated this code included: 
(i) general textbooks, 

(ii) general fauna e.g. wild life conservation, 

evolutionaury studies, 
(iii) general methodology e.g. X-rays, 

mathematical calculations, 
anatomy and histology, 
serological techniques, 

(iv) general biology e.g. enzyme activity, 

immunity, 

nutrition, 

disease, 

pharmacological testing. 
Our conclusions regarding the use of codes for Chordata and Vertebrata 
are as follows: 

(a) No clear distinction could be made between the general topics 
discussed in either group. 

(b) Topics were applicable to a wide range of animals; the titles or 
abstracts frequently contained terms such as: 'domestic' or 
•laboratory' animals, 'fauna', 'wildlife', 'living organisms'. 
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•vertebrates^ without further specif i .-i;:.: Lo i, 

(c) An article was allocated either the cod^? for Vertebrata, or the code 
for Chordata^ out never both. Furthermore, the article was never 
allotted aiiOther of the general BioSystematic codes e.g. SYS85700 
(Mammalia) . 

(d) Of the ^S^ references assigned the code Tor either Chordata or Vertebrata, 
M (7%) named particular animals either in the title or among the 
enrichm<^nt terms and were also indexed with the appropriate, more 
specific, BioSystematic code. For example, the following articles 

were assigned both a general and a specific code: 

"Studies on deep- freezing of boar semen. Part III. Additional 
effects of hyauLuronidase on livability of boar spermatozoa" 
(£L» No. 20, Abs.1i255(^) 

"Uterine weight and pituitary luteinizing^' normone content in the pseudo- 
pregnant hamster " 
Cil, No. 20, Abs. 113017) 

"Regeneration of the African Long Fish Proopterus" 
No. 20, Abs. 11 1750) 

(e) A further eight references {$%) appeared to deal with clinical studies 
which should have been assigned the code SYS^o2l5 (Hominidae). These 
references, which would have been missed by a profile designed to 
retrieve aspects of human medicine, were: 



abstract III4O30 - 


enriched with 'human*, but SYS36215 not allocated 


abstract 113339 - 


dealt with clinical experiments 


abstract 11i|121 ^ 


dealt with 'Clinical aspects of tiAmors 


abstract 11M20 - 


dealt with malignant tumors in the USSR 


abstract III4273 - 


dealt with leukemia cases 


abstract 111>52 - 


dealt with the clinical use of the fibroscope 


abstract 113272 - 


dealt with heart disease 


abstract 113930 - 


dealt with drug poisoning 
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^» Overlap between CROSS and BloSvatematlc codes 

There are several code headings which appear in both CROSS and the 
BioSystematic indexes. For example: 

a) Micro«organlamat CRS29500 and SYSOIOOO 

b) Paleontolofy 



Paleobotany 

Paleobiology 

Paleozoology 



CRS^OOOO and SYS29000 
CRSbl|500 and SYS27000 
CRS630OO and SYS3100O 



It is understood that current BIOSIS editorial policy is that CROSS codes 
are assigned on the basis of the subject matter of the paper, and the 
BioSystematic codes are allocated according to the organisms mentioned, 
a) Micro-organisms 

A profile comprising the CROSS code CRS29500 and the BioSystematic 
code SYS01000 was run against BA Previews ^1(23). Of the 91 references 
retrieved, it was found that: 

63 articles were coded with SYSOIOOO only 

28 articles were coded with CRS29500 of which 23 were also 
coded with SYSOIOOO, 

It appeared that the CROSS code was used where the paper discussed a 
fundamental activity of micro-organisms, e.g. enzyme formation by bacteria, 
microbial action in cereal digestion. In the 23 articles which had been 
given the BioSystematic as well as the CROSS code, named micro-organisms 
were mentioned in the enriched title. 

The BioSystematic code SYSOIOOO was used to indicate that micro- 
organisms (usually unspecified) were relevant to some aspect of the article. 
It was applied to articles in which general terms such as 'microbial' or 
'microflora' were used or in which the reference to micro-organisms was 
inferred. It was also used in conjunction with BioSystematic codes for 
named bacteria, fungi, etc, usually when these were being used to 
demonstrate a general principle. 

b) Paleontolog y 

A profile consisting of the three CROSS codes and the three BioSyBtem- 
atic codes concerned with paleontology was run against two issues of 
BA Previews , i.e. ^(21+) and ^(02). The profile retrieved 137 articles 
which had been assigned one or more of the paleontological codes. In 121 
ERXc °^ articles it appeared that these codes were allocated according 
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to the following*, principles : 

(i) che appropriate BioSystematic code was allocated whenever an .rganism 
was mentioned in the title, enrichment terms or abstract of a paper 
dealing with a paleontological topic. Specificsdly: 

SYS31000(Paleozoology) was assigned to 11^ papers referring to 
fossil animals 

SYS29000 (Paleobotany) was assigned to 20 papers referring to 
fossil plants 

SYS27000(PaLLeobiology) was assigned to only four papers. 

(ii) the appropriate BioSystematic code was accompanied by the equivalent 
CROSS code, i.e. 

SYS31000(Paleozoology) was assigned with CRS63000(Paleozoology) 
SYS29000 (Paleobotany) was assigned with CRS50000(Paleobotaiiy) 

(iii) the only paleontological CROSS code to appear without the corresponding 
BioSystematic code was CRSoUSOO (Paleobiology) which was used for 
general paleontological principles e.g. stratigraphy, analytical 
techniques. 

Of the 16 articles (out of 13?) which did not conform to these principles: 

(i) 3 had no indication in either title or abstract as to why SYS29000 
or SYS31000 had been added (the original paper was not examineed), 

(ii) 10 had no CROSS code corresponding to the BioSystematic codes which 
had been allocated, 

(iii) 3 had no paleontological BioSystematic code corresponding to the 
CROSS code. These were: 

'•The skin of Egyptian mummies. A study in survival" = 
histological techniques climate 
(£L» N0.2U, Abs.l3UUl2) 

"Plan.;tonic foraminifera differential production and 
expatriation off Baja, California** = Mexico iOumnerform 
fossils 

(jil, No*2U, Abs. 13^307) 
# 

''Remarks on the fi'enera Eucalycoceras and Protocanthoceras 
ammonoidea" = Pseudocalycoceras new genus fossil 

(ii, No.2li, Abs.13^3 'U) 
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C. RECOMMEINLA TIONS FOR nsiNQ BIOSYSTEMATIC CODES 

There are 18 codes describing categories of micro-organisms, 
393 codes describing categories of plants, 

325 codes describing animals (including 26 extinct 

categories), 
3 codes describing categories of paleontology, 
2 codes describing categories of pesticides. 

l-he studies described above .liid..oate that BioSystematic codes are 
allocated with a high degree of consistency to any specific organism or 
group of organisms which is mentioned in the enriched title or the abstract 
of the papey. In the construction of a search profile, the decision 
whether to use the specific name or the BioSystematic code as a search term 
will depend on the number of organisms which are of interest, and the amount 
of information required about them. 

1. Phylogenetic considerations 

a) BioSystematic codes are more suitable for the retrieval of broad 
categories of organisms such as: 

Phyla, e.g. Protozoa 

Classes e.g. Mammalia 

Orders, e.g. Marsupiala 

Families, e.g. Canidae 

b) If only one or a few genera or species are required, it is preferable 
to search for the generic or common names rather than for the BioSystematic 
code, which would retrieve much unwaiiled material. For example, BA Previews 
^(12) includes 1+8 articles coded 3ySo5j)u6 (Anura) which ;overs all toads 
and frogs. Only 13 of these articles refer to the genus Rana (frog) but 
there is no BioSystematic code specific to this genus. Thus, to retrieve 
only those articles which refer to Rana . a word search is preferable. 

c) When only Homo sapiens is of interest, the code SYS86215 (Hominidae) 
should be used rather than a list of all the possible synonyms for humans. 

2. Sub.iect range of BioSystematic codes 

The range of topics covered by articles which were coded with selected 
BioSystematic codes are summarised below, 
a) Codes for Viruses 

These were appLied to papers dealing withj 



ERIC 



- 165 - 



olassif ication of viruses 
disease etioloe^ 
culture growths 

biochemical reactions and metabolism 

antiviral agents 

carcinogenesis 

diagnostic pathology 

DMA replication and transcription 

biophysical techniques for identification 

b) Codes for Coniferopsida 

These retrieved papers dealing with: 

systematics and classification 

drugs derived from conifers 

palynology 

forest ecology 

wood products e.g. sawdust 

plant diseases affecting conifers 

allorjles caused by conifers 

their use as animal food 

c) Codes for Echinodeimata 

As well as papers concerning systematics, physiology, etc, these codes 
also retrieved articles concerning: 

embiyology using sea urchin eggs 
new species and locations 
population explosions (Crown-of- thorns) 
fossil finds and their classification 
their use as food (beche-de-mer) 

d) Codes for Suidae 

Papers retrieved dealt with: 

experimental surgeiy 

substances derived from porcine tissue 

porcine cells in tissue culture 

enzyme chemistry 

testing of food .supplements 

.jerm-free animals 

quality of pork and bacon 

e) Codes for Vertebrata and Chordata 

A general BioSystematic code (vertebrata, chordata, micro-organisms), 
unt no specific code, is often assigned to papers concerned with ^jeneral 
nethods, techniques and theoretical models. Searches for these general 
topics should not therefore be restricted to retrieving articles which 
have been assigned the 3io3ystematic code for a specific rganism. In 
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partioultr some articles oonoexning general applieatlons of ollnloal medioine 
axe assigned the general oode (vertebrate or chordata) but not the code 
for Yomm (SYS8621^). 
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Appendix II 3 

SPECIMEN PROFILE AND PRINTOUT OF REFERENCES 
FROM BA PREVIEWS SEARCH 

1. SPECniEN PROFILE Profile No. W65 

Statement of interest ; Fate of fertilizer nitrogen applied to soil, with 
special reference to leaching and denitx-ification, and methods of 
nitrogen ansdysis. 
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BREMNER J 
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CHATT J 








W6$01AO200 


KOLENBRADER G J 
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Wb50lTO1d0 


NITR0GEN-1$ - 


C 




W65;01TO1b0 


NATURAL ABUNDANCE 


D 




W6501TWD/l$0 


♦NITRIFICATION 


E 




W6501MA 


CRS5".'^05 


Soilsj Physics and Chemistry] ] 






W6501MA 


CRS52807 


Soils: Fertility studies '-■ ) 


P 




Wb$01TWS/l2$ 
W6'501TA 


NITROGEN* 
FERTILIZE 
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W6$01TA 


FIXATION 








W65;01TA 
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SOIL* 
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CARBON DIOXIDE 
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NITROGEN* 
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CARBON DI 


OXIDE 


J 
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*AEROBIC* 




K 




W6$01MA 


CRSi40000 


Soil Microbiology 
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W6$01MWD/$0 


MAJ$2u0$ 


K Oils: Physics and Chomictry ^' 
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STm$280$ 


_Soils: Physics and Ch'-miJatry .i 
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External effects: physical and 
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PLTIIP* 
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EXTRACT* 






Wb$01TA 


MEASURE- 








Wo$01TA 


SAMPLING 




P 




Wo502MN 


MA J $2$ (Agronomy 



Author terms 



Word tc-^TOS 



CROSS codes 



Primaiy CROSS code 
Secondaiy CROSS code 



Note 

Tne left-hand column definoG the logical relationships between terms. Thus, 
if v#e designate groups of tenns as A,B|C as shown, then the logic required 
by the profile for retrieval of a reference isj- 

A or 3 or C or (D and E) or (F and G) or (H aiid l) or (vT and K) or (L and M) 
orTN and O) but not P. 

Thp descending l'->rm weights aro used to ordor th- output. 

The *'NOT** logic'* ensures that a reference allocated the printing assignment 
:*jr A,;r.non;y (MAJ^J2p) will not be retrieved* 

O 
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MPLE OP RSgSRmGE PHmTOUT 
W6S 

BA VOU 52 NOt 22 DATE 24/01/71' 

IT£MS PRINTED i5 

RCMAirUNC 999994 



(ilORUANo C, OE PAiCALE C# ESPOSITO R# 

liALE^iTRIERl C# CITTADINI 0. 
)S^^ THE DISTRIDUTION 0^ NITR0Cr.N-j5 IN THE AMINO- 
ACIDS 0^ HEMO CUOBIN FrqM NORMAL SUBJECTS AND 
THOSE WITH UREMIA TREATED WITH NITROCEN-JS Idlk 

BIOCHIM APPU 

15 (6), I96G 392-336, 

NUMBER 123028 



ACARWAL A SINCH B R# KANt'HiRO Y» " 
IONIC EFFECT OF SALTS ON MINERAL NITROGEN 
^■'ELtASE IN AN ALLOPHANIC SOJLsUSA ORCANO 
INOKGANIC COMPLEX SULFATES CHLORIDES 
MITKiFiCATlOfi ELECTROLYTE MICROBIAL 
RESPIRATION FERTILIZERS 

SOIL SCI SOC AM PROC 

39 (3), 1971 494-457» 

ARTICLE WEIGHT 150 - numBER 127999 

NITRIFICATION, NITROGEN, SOILt 

DUDOREV M A# CHOBIT KO G L« 
APPLICATION OF BASIC FerTuUER IN RAISING 
PlNUS-StLVESTRlS-G SEEOL I NCSsN I TROCEN 
PHOSPHORUS ORGaNO MINERAL NUTRITION HUMUS 

BYUIL VSeS NAUCHNO-ISSLED INST AGROLESMEL I OR 

5 <57>. 1969 TRANS, 40»43# 

ARTICLE WEIGHT 125 NUMBER 124709 

NITROGEN* 

LANG N J# FAY P. 
THE HETEROCYSTS oF BLUe^GREEN ALGAE PART 2 
PETAILS OF ULTRASTKUCTURE«ANABAENA-CYLINDRICA 
POPE CHANNEL MICRO Pl ASMOOISISMaTA PLASMALEMHAS 
THYLAKOIDS- MEMBRANES NITROGEN FIXATION 

PROC R SOC LONO B BIOL SCl 

179 < 1051 )• 1971 193»203, 

ARTICLE WRIGHT 125 NUMBER 125146 

NlTROGENt 

WANFK W# ONDRACEK (,# HAMPL J« 
COVAlENT NITROGEN «»HOSPhORUS COMPOUNDS AS 
SOURCE OP PLANT MUTR I T I ONiOATS-M FERTILIZER 
CHEMICAL PHYSICAL PROPERTIES 

I PFLANZfcNE»:NAEHH BOOENKD 

129 (3), 1971 169-100, 

ARTICLE WEIGHT 125 NUMtiEK 127917 

NITROGEN* 



3. TERM FREQUENCY LISTING 
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Appendix II Ij 

Experimental Information Unit 
7 Keble Road« Oxford 

EXPLANATION OF BIOLOGICAL ABSTRACTS OTTTPUT (1970^ 

The first sheet of your output consists of your Profile Printout * 
showing the logical sequence of teiaas and codes, grouped in parameters, 
and indicating the weight assigned to each. For subsequent computer 
runs of this profile, you will not receive a profile printout but if 
your profile is amended, i.e. a new profile is established, you will 
receive a printout of this the first time it is run. 

The second sheet shows the Search Term Frequency , i.e. the number 
of times one of the terms or codes in your profile produced a 'hit'. 

. The remaining sheets are your Printout of references for one issue 
of Biological Abstracts or BioResearch Index. 

Example! On the specimen Printout attached, the following information 
is given I - 

BA Vol.51 N0.O3 Date 1^/03/70 

(This is the printout for Biological Abstracts, Volume ^1, Issue No. 3, 
which was run on ^$ March 1970) 

Items printedi 16 

(16 references were retrieved) 

Joumetl reference (examplt*) 

TKOCZ, C, KDHN, K (1) 

The formation of triple helical collagen molecules from alpha 1 or 
alpha 2 polypeptide chains (2) = calf rat skin (3) 
Eur J Biochem {k) 

Article weight 100 (9) 

Collagen^ ''^^ CRSIOO6U CRSIOSOb^'"''^ Number 12363^''^^ 

(1) Author(s) 

(2) Article title 

•'3) 'Enrichment ♦ words added to title by BA editors 



Explanation of Biological Abstracts Output 

(U) Abbreviated journal title 

(5) Volane 

(6) Issue 

(7) Year 

(8) Pages 

(9) Weight of this article, i.e. sum of weights of terms and codes 
which have produced a 'hit' 

(10) Term in profile which has ^^roduced a 'hit' 

(11) CROSScode in profile which has produced a 'hit' 

(12) Abstract Number in Biological Abstracts or^ 
Citation in BioResearch Index 

CROSScodes and BioS.vstematic Codes 

The CROSScode numbers correspond to the subject area headings in 

Biological Abstracts. The paper is cross-indexed and it is possible oo 

distinguish uetween three types of CROSS jodes:- 

CRS: Subjects in the CROSScode list, representing all 

subject areas covered by the article. The codes which 
appear in the prirted CROSS-Index in Biological Abstracts 
are differentiated as: 

MAJ: The principle subject area, i.e. that under which the abstract 
can be found in the printed Biological Abstracts. 

SUB: Important subject areas to which the articit? is cross 
referenced. 

N.B. If MAJ and SUB are not specified and only CRS appears in the profile, 
all three types will be retrieved. 



September 1^70 
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A. Academic 



Appendix III 1 
TABLE: LIST OP ESTABLISHMENTS 



Type 


Establishment 


Location 


No. of 






Profiles 



1. Oxford University 
SCIENCE: Agriculture 

Blochemlstzy 

Blomathematlcs 

Botany 

Forestry 

(jenetlcs 

Geology 

Physiology 

Zoology 

MEDICINE: Human Development 
Obstetrics and 
Gynaecology 
Ophthalmology 
Pathology 

2. Nottinflfaam University 
SCIENCE: Botany 

Phsirmacy 
Zoology 



Oxford 



2 
8 
2 

S 
2 

3 

2 ) 65 

8 

13 
2 ) 

12 

5 

1 



Nottingham 




AGRICULTUIiE: 



Agricultural 
Economics 

Applied Blo- 
chemlstzy 

Physiology 



ERIC 



3* Queen's University of 
Belfast 

Blochemlstiy 
Medical Biology 

!+• CambrlcUge University Cambridge 
Geology 
Physiology 

5. Dublla University Dublin 
Colleg;e 

Botany 
Medicine 

5. East Anglla University Norwich 

Biological Sciences 

7. Hatfield Polytechnic Hatfield 
Biological Sciences 

6. Liverpool University Liverpool 

Botany 



SuttnBcnnli%;tai 2 ) 62 
Belfast 



5 
3 
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Type 


Establishment 


Location 


No. of 
Profiles 












(contd) 


9. 


London University. 
Middlesex Hospital 
Medical School 
Biology 


London 






10. 


Loughborough University 

Chemistrv 
Ergonomics 


Loughborough 


1 ) 
h ) 




11. 


Manchester University 
Biological Chemlstiy 


Manchester 






12. 


Sheffield University 

Anatomy 

Pharmacology 

Physiology 


Sheffield 


2 ) 
2 ) 
h ) 




13. 


University of Wales. 
Welsh National School 


Cardiff 





of Medicine 



B. Independent ^. Animal Health Trust 

Research Estab-^ « ^ r.^ ^ 

2« Marine Biological 

Association of UK 



lishments 



C. Government 
a) Home Office 



3» Royal Society 



Central Research 
Establishment 



Nevnnarket 

Plymouth 
London 



b) Kin. of Agric* 1» Centra?. Veterinary 
Fisheries and Laboratory 
Pood 

2. National Agricultural 
Advisory Service 



3. Plant Pathology 
Laboratory 

U» Royal Botanic Gardens 

5* Sea Fisheries Research 
Laboratories 

c) mn. of 

Technology Toriy Research Station 

d) Agrlc. Research 

Council 1, Agricultural and 

Horticultural Research 
Station 



Aldeimaston 

Veybridge 

Cambridge 

Reading 

Derby 

Harpenden 
Kew 

Lowestoft 
Aberdeen 



6 

3 

9 
1 



2 

10 

2 
2 

11+ 

3 
1 

2 
3 



Long Ashton 



6 
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Establishment 


Location 


No* of 
Profiles 


(contd) 
d) Agrio. 
Research 
Council 


2. 
3. 


Institute for Research 
on Anlsial Diseases 

Meat Research Institute 


Newbury 
Bristol 


12 

5 




h. 


Natloned Institute for 
Research in Dairying 


Reading 


9 




c 

?• 


National Vegetable 
Research Station 


weiiesDoume 


1o 




6. 


Radiobiological 
Laboratory (Letcombe) 


Wantage 


7 




7. 


Rothamsted Bxperimental 
Station 


Hazpenden 


11 




8. 


Unit of Muscle Mechanism 
and Insect Physiology 


Oxford 


1 




9. 


Weed Resefiuroh Organisation Oxford 


h 


Research 
Council 


1. 
2. 


Radiobiology Unit 

Unit for Metabolic 
Studies in Fsychlatiy 


Harwell 
Sheffield 


h 

2 


f ) Natural 
Environment 
Research 
. Coiincil 


1. 
2. 


Fresh-Water Biological 
Assoc. of the U.K. 

Nature Conseirvancy 


Windermere 1 
Abbots-Rip ton 11 


g) EIRE 9 Dept. 
of Agric. and 
Fisheries 


Fisheries Division 


Dublin 


1 


h) EIRE, Institute 
for Industrial 
Research and 
Standards 


Industrial Research 
Centre 


Dublin 


1 


i) General 


1. 


Trent River Authority 


Derby 


5 




2. 


British Museum 
(Natural History) 


London 


5 


D* Industry 


1. 


Boots Pure Drug Co. Ltd. 


Nottingham 


5 




2. 


Fisons Pharmaceuticals 


Loughborough 


20 . 




3. 


Imperial Metsd 
Industries Ltd. 


Biiminghajn 


1 






May and Baker Ltd. 


Dagenham 


3 
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Type 


Establishment 


Location 


No. of 
Profiles 


D. 


Industry 
(contd) 


5. 


Plant Protection Ltd. 

(ici) 


jsrcic iui6 X X 








6. 


Lord Rank Research 
Centre 


High Vycomb 


e 6 






7. 


Wellcome Research 
Laboratories 


Beckenham 


11 






8. 


Harry Wheatcroft and 
Sons Ltd* 


TIo 1 1 Ingham 


1 
1 


F. 


Naturalists 




Mr. Davies 
Dr. Kent 
Mr. Sawver 
Dr. Veal 


Nottingham 
Nottingham 
Newport 
Nottingham 


1 
1 
1 
1 


G. 


Info mat Ion 
Centres 


1. 
2. 

• 


Bee Reseeorch Assoc. 

Biodeterioration 
Centre 


Gerrards Cross 1 
Blimlngham 1 






3. 


Biomedical Information 


Sheffield 


3 








Commonwealth Bureau of 
Animal Health 


Weybrldge 


1 






5. 


Commonwealth Bureau of 
Helminthology 


St. Albans 


3 






6. 


Commonwealth Forestry 
Bureau 


Oxford 


1 






7. 


National Documentation 
Centre for Sport, P*E. 
and Recreation 


Belfast 


2 






•j. 


Project PAIR (MRC) 


London 


U 








Total no. of 'Profiles 


392 
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Appendix III 2 

Experimental Information Unit 
7 Keble Road. Oxford 



BA PREyiEVS EXPERIMENT IN THE UK 



1. Introduction 

BA Previews consists of a magnetic tape for each semi-monthly 
issue of Biological Abstracts and each monthly issue of BioResearch 
Indexi i.e. a total of 36 tapes per year. 

With the exception of the actual text of the abstract, all the 
information contained in the main text and the indexes of the printed 
Biological Abstracts and BioResearch Index is now available on tape. 
The tapes are distributed approximately one month earlier than the 
corresponding printed issues of the journal. 

The annual lease of the BA tapes is )«3bOO, plus the cost of the 
tapes. 

2. Purpose of the Experiment 

The Office for Scientific and Technical Information (OSTi), a 
section of the Department of Education and Science, has been offered a 
yeax'8 free supply of BA Previews tapes for use in the UK and has awarded 
grants of £1U,0U8 and C^b^O to the Universities of Oxford and NottinAam 
for a two-year investigation of the use of the tapes. 

The results should provide a basis for deciding vdiether a service 
based on BA Previews should be set up for users of biological information 
in this country. In this context, the following aspects of the service 
will be investigated:- 

(a) Journal coverage, degree of currency, logical organization 
of the indexes and the allocation of index words and codes, 

(b) The best way of formulating computer-usable profiles 
representing user interests. 

(c) The most acceptable foim for a BA Previews service, 
i.e. Current Awareness only, or Current Awareness and 
Retrospective. 

(d) The relative performance of a BA Previews service compared 
with other mechanized and conventional services covering 
the same fields. 

* 

In addition, it is hoped to obtain some general information on the 
pattern of information needs and usage among workers in biology and the 
subject fields covered by BA Previews. 
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3* Staffing 

Three liaison scientists hav^ been appointed, two at Oxford and 
one at Nottingham* 

OXFORD 1, Miss Margaret Williams, B*Sc. , Dip. Lib. , A.L.A 

Experimental Information Unit ^ Botaiiy and Zoology graduate from the 
7 Keble Road, Oxford 0X1 3Qj4 University of Leeds, formerly Division 

Librarian at Imperial Chemical Industries 
Tel J Oxford 521+27 Limited, Pharmaceuticals Division, and with 

varied experience in industrial and 
academic library and information work. 



Tel: 56101, extn. 3lU3 



2. Miss Barbara Smith, M.Sc, a Zoology 

graduate from the University of Melbourne 
with research experience in vertebrate 
reproductive physiology, and formerly 
medical abstracter at Derwent 
Publications Ltd. 



NOTTINGHAM 

Department of Botany 
University of Nottingham 
Nottingham NG7 2RD 



yirs Janet Stow, M.Sc, a sandwich 
course gradua-ce in Applied Biology 
from Salford University, with research 
experience in cell culture. 



Their task is to build up a limited population of users, who will 
receive regularly a printout selected by the computer to meet their 
individual interests. The service is provided free in exchange for 
feedback from the users about its performance. 

Within restricted areas around Oxford and Nottingham, it is hoped 
to interview and offer participation to all potential users of the 
literature covered by BA Previews. This includes scientific papers, 
books and conference proceedings in a broad area of biology, as well as 
non-clinical medicine, a^icultural sciences, biochemistry and pharmacology. 
It is proposed to recruit not only university staff and research students, 
but also technical college staff, workers at government and industrial 
research stations and in industry and some school-teachers and amateur 
naturalists. 

The role of the user in the exDeriment 

(a) First interview 

If you would like to participate in the experiment you will be asked 
to have an interview (lasting apT)rrjximately ^y hours) with one of the 
liaison scientists. This will commence with the completion of a short 
questionnaire, which will help us to ascertain your background and 
literature searching practice. We shall then ask you for a statement 
of your research interests, from which we will proceed to compile a *user 
profile* . 

The user profile consists of a logical co-ordination of v/ords and any 
of the following categories of codf^s:- 

(i) Single or compound words: Appearing in titles of relevant articles, jV 

Added by 3A to enrich titles 

(ii) CROSS Index Codes 
(iii) BioSystematic Index Codes 
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(iv) Authors 
(v) Journal Codens 



In the computer searching, the terms (or codes) in the profile are 
matched against the record on the BA Previews cape and a printout of 
relevant ?ei'erences is produced from the ♦hits^ 

Bach month you will receive three printouts, two corresponding to 
the fortuii^tly issues of Biological Abstracts and one to the monthly 
issue of BioResearch Index* Accompanying each printout will be a small 
feedback card, on which you are asked to mark which references are 
Inportant or Relevant and return it to the liaisou scientist* The 
following definitions should assist you in making a decision:- 

IMPORTANT: a reference directly related to your research interests 
^ich you will read as soon as possible, i«e« it is 
essential that you know of it* 

RELE7AljT: a reference which though not important, is of some 
interest to you, i^e* you are glad that you were 
notified of it* 

The iii5)ortant category should be interpreted restricti/ely and the 
relevant category generously. 

You may already know of (or have read) a reference before it appears 
in the output. However, your relevance judgments should be independent 
of whether a }^ference is familiar to you or is * out-of-date ♦ . 

(b) Second interview 

When you have received a few initial printouts, you will be contacted 
by the liaison scientist for a seuund interview (approximately ^ hour) to 
discuss the effectiveness of the search, so that any necessary amendments 
to your profile can be made. If, however, you consider the search to he 
of little value no your work, it would be advisable for you to discontinue 
as a user. 

It is hoped that the requisite number of participants will have been 
acquired by the late summer, so that the experiment can become fully 
operational. It will then run for one year, at the end of which there 
will be a final interview w^th the liaison scientist to assess the value 
of the experiment to the user. 



[The specimen profile and the aimotated sample of computer printout, ithidh 
were distributed to users with the above oirculri.r, are not included in 
this appendix. Samples of both a profile and a printout appear in 
Appendix II. 3]. 
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Appendix III "=1 



BA PREVIEWS 



QITESTIQMAIRE 



1 . Name J 



Profile No: 



2. 



Address: 



Type of organisation 



(a) University or College (incl. Teaching Hospital) 

(b) Independent Research Establishment 

(c) Government Establishment 

(d) Indus tiy 

(e) School 

(f) Free-lance naturalirt 



h* No, of years resesirch experience 
$, Background 

Are you scanning/searching exclusively for your own work? Yes/N 
If No « please expand: 



6, Manual literature sestrching practice 



(iii) 
(iv) 
(v) 
(vl) 

(vii) 



(a) Do you see certain primary journals regularly? 



Yes/N 




(viii) 
(ix) 
(x) 
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(b) Of the following list of secondary journals, which, if any, 
do you search regularly: 



Title 


Regularity 


AvaiiaDiiity 




C 


R 




Biological Abstracts 








Chemical Abstracts 








Chemical Titles 








C.B.A.C. 








Current Contents 








Index Medicus 








Intemat. Abs. Biol. Sci. 








Bulletin Signaletique 








Excerpta Medica 








British Medical Abstracts 








Science Citation Index 








Zoological Record 









Regulaxity : C = Current awareness purposes 
R = Retrospective Searches 
J = Prime Tool 
X = Occasional Tool 



Availability t L =: Own Library 
Circ. = Circulation 

B = Nearby bxxilding 
D = > 10 minutes away 

(c) Do you use any specialised s econdary journals (e#g# 

Microbiol# Abs#) including Indexes, Bulletins Yes/No 

If yes , specify: 

(d) Is your searching adequate Ygs/No 
If no 4 give reasons: 



ERLC 



Mechanised Services 



(a) Have you used any of the following mechanised services: Yes/No 

(i) Medlars - Monthly 

Retrospective 

(ii) ASCA 
(iii) CT 

CBAC 
POST-J 
(iv) Ringdoc 

(b) If yes , have you an existing profile? 

(c) If yes , has this eliminated the need for manual 



searches of the secondary journals? 

(d) If yes , has this redu^'^d the number of primary 
journals you now scan? 

If no , to all, 

Are you familiar with an IR computer printout? 

Use of Biological Abstract ?^ 

Do you use Biological Abstracts? 



Yes/No 
Yes/No 

Yes/No 



Yes/No 
Yes/No 



If yes , 

(a) Do you use BA for 

(i) Current Awareness Regularly/ Spasmodically/N^er 

(ii) Retrospective searches Frequeut-ly/ Infrequently/Never 

(b) Indexes: do you consult: 



: c I R 


AUTHOR i 




B.A.S.I.C. 1 


i 


Cross-code j 


1 


Taxonomic co./? ' 


! i 
1 1 1 


Appropriate subject group 


1 1 



Comments on design and presentation: 



(c) Do you think the journal covera<;p is adequate? Yea/iu; 
If no , what title? would you like to ;:;ce inoludod? 
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(d) Do you consider the abstracts are sufficiently 
infoimative? Yeg/Uo 

(e) Does the time-lag between publication of an article 
in a pr.'.mary journal and its appearance in BA render 
the latter inadequate as a Current Awareness Tool 

yo^^-" Yes/No 

If no . 

Is your non-use of BA attributable to: 

(i) No subscription to it by organisation 

(ii) Inaccessibility 

(iii) Do not like foimat of indexes 

(iv) Do not consider Journal coverage adequate 
for own subject field 

(v) Rely on another service 
i.e. 

(vl) Do not find any need for abstract services 

(vii) Tend to rely on "Invisible Colleges", 
i.e. personal contacts with co-workers 
in own field. 

(viii) Other, specify: 
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Appendix III L 



TABLE: GROUP AND SHARED PROFILES 



Total no. of group and shared profiles written 

No* of group and shared profiles deleted J;ine 1971 

No» of group profiles participating in the evaluation 

No# of shared profiles i.e* only one person was inter- 
viewed, acting on behalf of a group 

No. of group profiles i.e. more than one person was 
interviewed 



hi 
s 
20 
12 

35 



Group 


Group 


Group 


i 

1 Group 


1 

Total 


of 2 


of 3 


of h 


: of>h 





Evaluees 13 

Non-e value es ,i 

Deleted profiles 1 

Total 22 



h 


1 


3 


2 


1 1 


1 


1 

0 1 


h 



18 
Mi 
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Appendix III ^ 

FEEDBACK CAM) 
(before October 1970) 





BA - PREVIEWS 


NAME: 




PROFILE NO. 


ISSUE NO. 


References 


Total No. 


retrieved: 






Important 




Relevant 


Comments : 





ERIC 



- 18/3 - 



FEEDBACK CAE33 
(after October 197O) 



BA - PREyiEVS 

NAME: 



PROFILE NO. ISSUE NO. 



References Total No._ 
retrieved: 

Major Value 



Minor Vc.^ue_ 

Comments : 
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Appendix III 6 



TABLE: ANALYSIS OP COMMENTS MADE ON FEEDBACK CARDS 
RETURNED BY A SAMPLE OF 100 USERS 



Comments 



No of Users 
making this 
comment 



A. PROFILE CONSTRUCTION 

(a) Retrieval of unwanted terms, codes or 
journals (specified): suggestions for 
removal 

(b) Retrieval jf unwanted subjects (unspecified) 

(c) Subsequent to amendment of profile 

(i) General satisfaction 
(ii) General dissatisfaction 
(ili) No difference 

(d) Subject interests not adequately satisfied 

(e) Subject interests adequately satisfied 

(f) Truncation of words resulting in retrieval of 
unwanted references: suggestions for removsd 

(g) Spelling error in profile (specified) 

(h) Additional terms (specified) for inclusion in 
profile: suggestions 

(i) Lack of underptanding of coding of profile 

(j) Retrieval of too many references: no 
suggestions for decreasing 

(k) Retrieval of specified unwanted authors, e.g. BROWN R 
retrieves BROWN EE 

B, VALUE OF REFERENCES 

(a) References from journals not scanned by user 

(b) General satisfaction with printout 

(c) Most of the MAJOR VALUE references alreadj' seen 

(d) Some of the references are very old 

(e) MAJOR VALUE references retrieved which had been 
missed by the user .In his own searching 

(f) Reference to a paper written by the user 



26 
22 

18 

h 
1 

13 
10 

10 
8 

6 
5 

h 
2 



10 
8 
8 
5 

k 
h 
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No. of Users 

Comments making this 

comment 



is) References not available from NLL 3 

(h) References inaccessible, but interesting 2 

(i) Query: Why was a certain reference retrieved? 2 

C. RELEVANCE DECISIONS 

(a) High proportion of relevant articles on printout, 

as most of them were from one highly relevant 8 
publication, e.g. Symposium, Conference proceedings 

(b) Difficult to distinguish between MAJOR VALUE and 
MINOR VALUE 



ii 



(c) Confusion between IMPORTANT and RELEVANT (i.e. 

IMPORTANT included with RELEVANT references)2 3 

(d) Some of the irrelevant references are of * fringe' 
interest: would like a third relevance category 

(e) Difficult to determine relevance from title only 2 
D. FORMAT OF PRINTOUT 

(a) Difficult to read 16 

(b) Misunderstanding of bibliographic details (e.g. Issue 
numbers of BA and BRI are in different sequences) 3 

(c) Easier to read than usixal, i.e. better quality ^ 
printout 

(d) Lack of understanding of BIOSIS notation (i.e. the 

addition of -D and -M to plant names to indicate 2 
di- or mono- cotyledons) 

(e) Part of bibliographic citation missing: full title 
requested 

(f ) Search term frequency list missing 2 

(g) Dislike of green-lined paper 1 

(h) Printout sheets give pagination but not profile no. ^ 
or issue no., so could become misfiled 
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Foot Notftfl tr> T a>>1a 
1 

made it. 



This figure refers to the number of individual users who made this 
comment^on a feedback card, irrespective of the number of Jies^hey 

^ eliminated by the replacement of IMPORTANT and 

categories, i.e mJO^ VALUE 

3 

To assist users in understanding the printout, the leaflet "F^m^fl«.+^«« 
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Appendix III 7 



BA-PREVIEWS PROJECT 



UNIVERSITY OF OXFORD 

Miss Margaret I Willianu. B.Sc, Dip.Lib., A.I. A. 

Miw Barbara Smith, M.Sc. 



EXPERIMENTAL INFORMATION UNIT 



7 KEBLE ROAP 



UNIVERSITY OF NOTTINGHAM 
Mrs. Janet Stow, M.Sc. 
(Department of Botany) 



OXFORI\ OXl .^Ql. 
Telephone: Oxti»ril 52427 



May 1971 



In my letter of 16 April, I mentioned that I should be writing 
later to enquire if you would be willing to take part in the detailed 
evaluation phase of our experiment 

The object of the evaluation is to investigate characteristics of 
che BA Previews service such as coverage of the literature, the currency, 
and the quality of indexing. 

In order to assist you in deciding whether or not you are prepared 
to undertake this work, I am enclosing a form which gives details of the 
experiment and the work required from participants. If there axe any 
points about which you are not cleau:, I hope you will not hesitate to 
contact me. 

Would you kindly complete the enclosed card as soon as possible, 
indicating whether or not you are willing to take part as an evaluee. 

Even if you are not able to act as an evaluee, we hope you will continue 
to return the feedback cards. I am sure you will appreciate that it is 
expensive to run an experiemental cor^rkuterl^ed information service such as 
BA Previews. If I do not hear from ^ ^ by 31 lay 1971 > I shall assume 
that you no longer wish to receive printouts and your profile will be 
deleted. 



Yours sincerely. 



Margaret Williams 
Senior Liaison Scientist 



ERIC 



3V-U 
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Experiaentai Infomation Unit 
7 geble Road. Oxford 0X1 3QL 



EVALUATION OF TH£ BA-PREVIE W3 MAGNET IC TAPE SERVICE 

t 

Evaluation of the BA-Previews service will take into account the 
follovfing factors: 

1 . The proportion of references retrieved which are relevant 

2. The proportion of references 'missed' by the service 

3. The cost of providing the service 

4. Acceptability to the user. 

A detailed analysis will be made of relevant references which were not 
retrieved or irrelevant references which were retrieved. It is hoped 
that this 'failure analysis' will indicate ways of improving the existing 
service. 

Information recuired from participants in our evaluation will be: 

1 . In order to identify the proportion of relevant references missed 
by the servi^^e, you will be asked to report relevant references found 
by your usual methods of searching the literature (e.g. Primary 
Journals, Abstract Journals, Titles Lists, 'Personal Communication*). 
This feedback will be for a maximum period of six weeks, during 
June - July. 

You will also be asked to identify all relevant references 
appearing in 4-o issues of the BA computer output and indicate 
whether you have already seen these references. This second set 
of feedback will be required during the period July to October. 
The exact period can be varied to take account of summer holidays, etc 
3. Some users will be also asked to make relevance judgments on 
an output fro-, another computer service such as Chemical Titles or 
MEDLARS. 

Special forms will be provide! for listing the relevant references, 
and detailed instructions will be sent out at each stage of the experiment 
In order to identity a reference it will only be necessary to quote first 
author, journal abbreMation, volume nunter, year and first page number. 



/continued 
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Evaluation of the 3A-?revieys Magnetic Tape Service (continued) 



We recogr.ise tnat the provision of this detailed feedback will 
involve you in so-ne extra work and in designing the experiment we have 
endeavoured to keep this to a minimum. The amount of work *ftiich will be 
requires of you shoul.i occupy not more than one hour of your time in any 
one week. If, after agreeing to ta.-:e part in the experiment, you find 
that the evaluation is taking up too much of your time, the period in which 
you send feedback can be reduced, e.g. reporting of references found by 
♦your usual uethods" oar. be limited to two instead of six weeks. 

At the end of the experiment all participants, including those not 
able to take part in the full evaluation, will be asked to complete a final 
questionnaire. This will provide an opportunity for you to comment on the 
merits of the service, and also on the methods which we have used for 
collecting data. 



May 1971 BV-$ 
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Evalviatlon of BA-PrevlevB 

* I an willing to take part in the evaluation phase \ | 
of the experiment L I 

□ 



* I am not willing to take part in the evaluation phase 
of the e:q)eriinent, but will continue to provide 
regular feedback 



* I wish to withdraw from the experiment 

* Please tick in the appropriate box 
Profile Not Names 



□ 



[Card which aocoiqpanied the circular letter BV $]» 



Ksr COPY mmii 



Appendix III 3 
TABLE J EVALUEES BY CATEGORY OP EMPLOYMENT 



Employment Category 


No. of 

i!jVdJ*Ue6S 


No. of 
rroriies 


A. 


Academic 


56 


60 


B. 


Independent Research Establishment 


8 ; 


) 


C. 


Government: 


• 






(a) Ministries, NERC etc. 


18 I 


18 




(b) Agricultural Research Council 


39 


ho 




(c) Medical Research Council 


2 


2 




Industry 


21 

> 


21 


i 

1 F 

1 


Naturalist 


' ! 


1 


\ G. 


Information Centre 


6 i 

i 

i 




1 

< 


TOTALS 


151 


160 1 



- 197 - 



Appendix III 9 



TABLEt E7ALUEES BY SPECIFIC SUBJECT CATEGORY 



ERIC 



Subject Categozy 


PlOe 01 


No* of 




i!iv cLLuees 


Profiles 


1« Agricultural Science 


9 


9 


2« Aqxxatlc Biology 




h 


3« Biochemistry 


15 


15 


1|« Biometrics 






5« Biophysics 


6 


6 


S. Cell Biology 


7 


7 


7« Ecology and Conservation 


6 


6 


8« Qoobxyology 


1 


1 


9* Endocrinology 


5 


5 


10« Entomology 


6 


6 


11* Experimental Psychology 


1 


2 


12. Pood Science 


h 


5 


13* Forestry 


3 


3 


^h• Genetics 


2 


2 


17« Lmnunology 


2 


3 


1o« Medicine 


3 


5 


^^• Microbiology 


9 


9 


10» Nutrition 


1 


1 


19* Palaeontology 


1 


1 


20. ParMitology 


3 


3 


21. Pathology/Forensic Science 


k 


ii 


22. Phaimacology/Toxicology 


7 


7 


23. Physioloarv 


13 


■4 n 

17 


2I4. Plant Pathology 


8 


8 


25. Plant Physiology 


20 


20 


26. Public Health 






27» Systematic Botany 


3 


3 


28. Systematic Zoology 


3 


3 


29. Veterinary Science 


3 


3 


3(i Virology 


2 


2 


TOTALS 


151 


160 



- 198 - 



Appendix III 10 



TABLE: MANUAL LITERATDBE SEARCHING: PRIMARY JOURNAL USAGE. 
LIST OP PRIMARY JOURNALS CITED BY 10 OR MORE USERS 



Journal 


No. of 

Citations 


Nature 


159 


Biochemical Journal 


72 


Blochimica et Biophyeica Acta 


51 


Journal of Biological Chemistry 


U7 


• 

Plant Physiology 


h3 


Jouxnfid of Physiology 


39 


Science 


36 


Lancet 


32 


Annals of Applied Biology 


29 


Journal of Ecology 


28 


British Medical Journal 


2? 


Joumfid of Experimental Botany 




Journal of Applied Bcolo^':'' 


25 


Joumfid of General Microbiology 




Biochemistry 


23 


Joumfid of Economic Entomology 




Proceedings of the Academy of Sciences of 
the United. States of America 


22 


Archives of Biochemistry and Biophysics 


21 


Phytopathology 




Planta 


20 


New Phytologist 


19 


American Joumad of Physiology 


18 


Analyticed Biochemistry 




Journal of Moleculfiu: Biology 




Plant Pa'ohology 




Veterirjiiy Record 




Journal of Bacteriology 


•7 


Journal of Immunology 




Journal of Marine Biology 





ERIC 
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No. of 




Citations 


^i\fo±\j±.\jg±Of irX oil boUbUIu 






16 


Journal of General Virology 






15 


Etidoorinology 


Jouxnal of Awi^nnl Ecology 




Journal of Endocrinology 




Plant and Soil 




Virology 




Biochemical and Biophysical Research Cosms, t|. 


Caziadian Journal of Botany 




Journal of Con5)arative Pathology 




iustralian Journal of Biological Sciences 


13 


Journal of Cell Science 




Journal of Reproduction And Fertility 




Journal of the Science of Food and Agriculture 


Plant Disease Reporter 




American Journal of Botany 


12 


British Journal of Nutrition 




Bulletin of Entomological Research 


• 


Immunology 




Journal of Agricultural and Food Chemistry 


Journal of Zoology 




Research in Veterinary Science 




Journal of Experimental Biology 


11 


Journal of Soil Science 




Journal of Ultrastructure Research 


11 


Parasitology 




Soil Science 




Weed Research 





American JoumaJ. of Obstetrics and Gynecology 
American Journal of Veterinary Research 
Annals of Botany 
Canadian Journal of Microbiology 
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BEST COPY AVAILABLE 



Journal 



No. of 
Citations 



Journal of Insect Physiology- 
Journal of Nutrition 

Transactions of the British Mycological Society 
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BA PREVIEWS PROJECT 



Experimental Information Unit, 
7 Keble Road, 
Oxford 0X1 3QJi 

February 1971 



Sear 

When we Invited you to participate In our eatperlment, we told 
you that we were offering you a free computerized Information service, 
^wever, we stressed the Importance to vs of a regular feedbfiwk on its 
performance • 

On checking our records, I find that I have not received a feed- 
back card from you since the one for BA Issue No. (run on 
). I should be very grateful If you would kindly fill In 
the outstanding cards and return them to me ets soon as possible, so 
that I can complete my records. 

In March, we hope to commence the detailed evaluation phase of 
our experiment, when It will be essential for us to have Information 
from our users. If I hear nothing from you before the end of February, 
I shall assume that the service Is of no potential Interest to you 
and you no longer wish to receive BA printouts. In this case, I 
shsdl be obliged to discontinue sending them to you. 



Yours sincerely. 



- 202 - 



Appendix III 12 



BA PREVIEWS PROJECT 

28th June 1971 

Dear 

I am viriting to inform you that, as you have indicated that you 
wish to withdraw from the experiment, your profile has now been deleted 
and you will receive no further print-onts. * 

However, I should be interested to know your reason for withdrawing. 
Accordingly, I am attaching a brief questionnaire, listing possible 
reasons and would be grateful if you would kindly mark the relevant oneCs)* 

A stamp addressed envelope is enclosed for your reply* 

Yours sincerely. 



End. 
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Reasons for wlthdrawlgg from thm 
BA-Prsvl ws Bcperlma^^ t 

.1. Bl-Prsvisws Servics has proved of no value 

a) Have already seen most of the references 

b) The references are too old to be of any use 

c) The coverage of BA-Previews Is inadequate for my research 

topic 

d) The print-out includes too many irrelevant references 
2. Rely on another service 

a) Abstracting, or Indexing Journal 

b) Library/information Service 

c) Invisible College 

d) Other (please specify) 

tTirJSstlfiS''*""''* """^ fd-back is too tine-coaniiOaf 

♦ 

to™lef P'ia^-o^t i» unacceptable (e.g. illegible, difficult 

5. Research topic has changed, rendering profile inadequate 

6. Leaving the U.K. for work abroad 

7. Other (please specify) 



fir!«iMiiJ!J* "rlKL withdrawing fro. 

the experiment, if there are further reasons for your dissatiifii^T*.,. 
With the service, please indicate these with a X <^"»*i»'*otlon 
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A SAMPLE OF USER REQUIREMENTS 

One line Bummariee of the statement of requirements of twenty users are 
given below. It must be emphasised that many requirements were more complex 
than can be indicated in a single sentence. 

1. Mechanism of action of antimicrobial compounds. 

2. Bivironmental physiology and psychology in humans, in particular the 
influence of heat, noise and li£^t. 

3. Soil physics, management, hydrology and aeration. 
1^. Salt and water balance in the lower vertebrates. 

5. Biology of the Tettigoniidae and insect eggs. 

6. Intestinal microorganisms in poultry; the effect of nycotoxins on poultry. 

7. Diagnosis of tropical diseases with particular reference to parasitic 
diseases. 

8. The digestion and absorption of proteins. 

9. Pectolytic enzymes. 

10. Animal waste management. 

11. Ruminant nutrition and metabolism. 

12. Factors affecting the growth and development of roots. 
13* Biological control of insect pests in soil. 

1U. Developmental genetics of Drosophila melanogaster . 

1^. The biochemistry of the normal and cataractous lens of mammiJs. 

16. Comparative studies of the biology of selected grasses. 

17. Soil and plant factors affecting the efficiency of herbicides in weed 
control. 

16. Preparation and use of anti-lymphocyte serum. 

19. Crop diseases and their chemical control. 

20. Effect of agricultural pesticides on wild life. 
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CONTENTS 

1 . INSTRUCTION SHEET BBGABDING PNECISION FEEBBACKt HELEViNT BEE'ERENCES 
RETRIEVED BY USE BA PREVIEWS SERVICE 

2. COCIINENT SAMPLE: INSTRUCTION SBEET 

3. INSTRUCTION SHEET AND FORM BV-6j REFERENCES POTOn) BY SEARCHING OP CUHREMT 
LITERATURE [Gave instruotions to users about the references identified 
by the user in the course of his nozmal current awareness searches. These 
references fozmed the BV-6 saaiple used for conrputing covera^^e and recall 
ratios]. 

k» ^struction sheets for the alternative services %diich were used to provide 
recall and coverage samples 

GHaaCAL ABSTRACTS CONDENSATES (CAC) PRINTOUT 
MEDLARS PRINTOUT 

ASCA SERVICE [request for a list of cited authors or references] 

ASCA SERVICE: CITATION SEARCH [instructions regarding output from 
the citation search] 

5. PROCEDURE FOR THE ANALYSIS OF COVERAGE SAMPLES 

[Methcl of determining whether or not a ref eresice from the coverage 
•ample was covered bjr BA or BRI] 

6. TABLE: DISTRIBUTION OF APPEARANCE DATES IN BA PREVIEWS OF REFERENCES 
FROM COVERAGE SAMPLES 

7. CURRENCY SPREAD IN BA/BRI FOR INDIVIDUAL ISSUES OF PRIMARY JOURNALS 

8. .TABLE: SERIAL PUBLICATIONS NOT COVERED BY BIOLOGICAL ABSTRACTS OH 

BIORESEARCH INDEX 



ERIC 
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Ebcperifflental Infoimatlon Unit 
7 Keble Road. Oxford. 0X1 -^QL 



BA PREyiEWS EXPERIMENT 

RELEVANT REFERENCES RETRIEVED 
BY THE BA PREVIEWS SERVICE 



INSTRgCTIOK SHEET 

GENERAL 

1. ?0T k isBuee of BA Previews, you will receive 2 copies of your 
printout but no feedback cards. One copy is for your own use and the 
second copy for you to indicate the relevance of each reference retrieved. 

2. In assessing the relevance of a reference, the original definitions, 
tdiich you have been using vhen completing your feedback cards, apply:- 

Ma.1or Value (KAJ): Very centrally related to your research 

Interests and could make a significant contribution. 

Minor Value (HIN): Of interest, and happy to have been informed 

of it, but of no great loss to have missed it. 

Irrelevant (x) i Of no interest 

INFORMATION REQUIRED 

3* Will you please mai^ each reference as:- 

MAJ (Major Value); MIN (Minor Value); X (irrelevant) 
U* References already known t In adc;tion, will you please put an S against 
any MAJ or MIN references if you already knew of the existence of this article 
before you saw it on your computer printout. 

5. Each reference should be marked in the outer margin, i.e. the lefthand 
column of references should be marked in the lefthand margin, the rij^thand 
column in the righthand margin. 

6. Examples t MAJ Article of Major Value: not already known 

MIN/S Article of Minor Value: already known 
X Irrelevant 

7. The output should be returned in the enclosed addressed envelope, as 
soon as possible. 

GROUP PROF TT.Tgg 

d. For ao)t profiles, we expect only one person to be assessing relevance, 
even thou^, in tome cases, several other research woz^ers (e.g. graduate 
students) may see the computer printout. However, at a few intervie%r8, more 
than one person attended and contributed to designing the profile and each 
has been making his own relevance decisions. If this is so, the following 
procedure should be adopted:- 

(i) Mark at the top of the output the number of people making relevance 

Judgements (e.g. 3 people) 
(ii) Each petrticipant should provide an independent set of decisions. 

Example ; If three people make relevance decisions, each reference will have 
3 marks, e.g. MAJ/S 
X 

MIN 



BV-7 
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Experimental Infoimation Uni"-. 
7 Keble Bo ad. Oxford. QX-| ^y . 



BA PBEyiEtfS PROJECT 
DOCUMENT SAM PLE t INSTRPOTION SBBEfF 

1. For the final stage of the evaluation, we are sending you a randooi 
sample of documents. 

2. By comparing relevance Judgements made on the BA Previews printout with 
those made on the actual document, we wish to determine the extent to which 
a research woricer is able to maki a coircect prediction of the relevance of a 
document on the basis of the title alone. 

3. The enclosed articles all appeared on one or other of the four duplicate 
printouts you received a few weeks ago and may or may not have been judged 
relevant by you. In this cfiLe, you were assessing relevance on the infonwitlon 
supplied on the printout i.e. Title, Enrichment words, and Bibliographic 
reference. 

U. After studying those articles, we should like you to indicate the 
relevance of each, without reference to yo\ir previous Judgement. 

5. Will you please maiSc at the top of each document t- 

MAJ (Major value) / MIN (Minor value) / X (irrelevant) 

6. The documents should then be returned in the enclosed addressed 
envelope, as soor as possible. 
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Experimental Infonuatlon Unit 

7 Keble Road, Oxford, 0X1 BA PREVIi]VS PROJECT 

INSTRUCTION SHT^Tr.T 
REFERENCES FOUND BY SEARCHING OF CURRENT LITERATURE 

A. GENERAL INSTRUCTIONS 

1. Please use BLOCK LETTERS when completing- the form. 

2. Please list up to 30 references, found during the next six weeks 
by your noimal methods of searching the currant literature, which 
are relevant (MAJOR or MINOR value) to the research interests your 
profile 13 intended to serve. 

3. The foims should be returned on or before the date given at the 
top of the foim. 

h. References may bo obtained from any source except Biological Abstracts 
or your computer printout. Possible sources will include abstract 
journals (e.g. MicroblolocT^- Abstracts) , titles lists (e.g. Current 
Contents;, original (primary) Journals or personal communication from 
a librarian or collea^e. 

N.B. References found In an abstract journal, other than BA, are of 
particular Interest for the experiment. If you use an abstract journal 
for current awareness purposes, please make a special effort to scan 
some of the irrent Issues during the next «ix weeks and at least one 
issue If the abstract journal Is published irort.hly or quarterly. 

5. We realise that some participants will find considerably less than the 
30 references which Is the maximum number for which space is allowed 
on the form. However, If you expect to find more than 30 references 
during the time allowed, give preference to: 

(i) references located in abstract rather than primary journals; 
Ui; references located in earlier rather thaxi later issues of 
primary joumsdsj 
(iii) MAJOR rather than MINOR references. 

B. INFORMATION REQUIREI^ ON THE FORM 



For references to journal articles give first author, journal 
abbreviation, volume number (or year), issuo number (if any) and 
• irst page number. 

References to conference proceedings, books, reports and other types 
of 'non-Journal » literature 3hould be specified by the nr^t author and 
sufficient additional information to enable us to identify the- 
re ference. 

Sou£Co: Indicate where you located the reference, e.^. original 
(primary) journal. Chemical Abstracts, Forestry Ab.- tract.:., a library 
bulletin, personal communication. 

Read: You are not being asked to read a paper before deoi iin.-: 
relevance. However, if you have raad it, please put c'-ui ' ' 
colUiMn. 



•X' 
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BA-PREVIEWS PROJECT 

It is most important that the attached forms are completed 
and returned by the specified date if at all possible. 
However, some participants may not be able to do 
literature searching at any time during the next six 
weeks owing to holidays, field work or for other reasons* 
If delay is unavoidable, please keep the form (BV-6) 
but return this card indicating the month during which 
you expect to be able to undertake the searching. 

Name: Profile No: • 

Month when searching 

could be undertaken 
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Experinental Inforaation Unit 
7 Keble Road« Oxford 

BA-PREVIEWS PROJECT 
CHEMICAL A BSTR ACTS CONDENS ATES (CAC) PRINTOUT 
INSTRUCTION SHEET 

GENERAL 

1. The next atage of our evaluation involvea a coapariaon of the 
eoTorage of BA-Previewa with MEDLARS (the aaffoetic tape veraion of 
Index Medicua) and Chemical Abstracta Condenaatea (the tape Teraion 
of Chaaical Abatracta) for thoae evalueea whoae aubject intereata are 
in the ^bioaedical or biocheaical field reapectively. 

2. For your own subject requiresenta, Cheaical Abatraeta Condenaatat 
(CAC) would appear to be the appropriate alternative aervice. We 

have therefore conatructed a CAC profile to coTer your reaearch iataraata 
and thia haa been run againat 4 weekly iaauea of CAC tapea. 

3. Two eopiea of the CAC printout are encloaed. One copy ia for yo«r 
own uae and the aecond copy for you to indicate the reloTance of aach 
refertnca retrieved. 

INFORMATION REQUIRED 

4* Will you pleaae aark each reference aat- 

MAJ (Major Value) / MIN (Minor Value) / X (irreltYant) 

.6. Each reference ahould be aarked in the outer aarginf i.e. the 
lefthand coluan of referencea ahould be aarked in the lafthand aargin, 
the righthand coluan in the righthand a.'^rgin. 

6* We do not^ require you to indicate whether or not you already knew 
of the exiatence of an article. 

7* The output ahould be returned in the encloaed addreaaed envelope, 
aa aoon aa poaaible. 

GROUP PROFILES 

8* If aore than one peraon ia aupplying relevance deciaiona, pleaae 
follow exactly the procedure outlined in our previoua inatruetion aheet 
for 'Relevant Referencea Retrieved by the BA-Previewa Service* (.BV-7). 
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Experimental Information Unit 
7 Keblfe Road. Oxford 

BA-PRBVIEW3 PROJECT 
MEDURS FRIWTOUT: INSTRUCTION SHEET 

QEKERAL 

1 . The next stage of our evaluation involves a comparison of the 
coverage of BA-Previews with MEDLARS (the magnetic tape version of Index 
Medicus) and Chemical Abstracts Condensates (the tape version of Chemical 
Abstracts) for those evaluees whose subject interests are in the biomedical 
or biochemical field, respectively. 

2. For your own subject requirements, MEDLARS would appear to be the 
appropriate alternative service. We have therefore constructed a MEDLARS 
profile to cover your research interests and this has been run against 

a 3-^Bonth MEDLARS file. 

5. Two copies of the MEDLAHS printout are enclosed. One copy is for 
yaur own use and the second copy for you to indicate the relevance of 
each reference retrieved. 

IMFORMATIQN REQUIRED 

4. Will you please mark each reference as:- 

MAJ (Major Value) / MIN (Minor Value) / X (irrelevant) 

5. Each reference should be marked in the outer margin, i.e. the lefthand 
colxunn of references should be marked in the lefthand margin, the righthand 
column in the ri^thand margin. 

6. We do not require you to indicate whether or not you already knew of 
the existence of an article. 

7. The output should be reti\-ned in the enclosed addressed envelope, as 
soon as possible. 

GROUP PR0FILP;3 

8. If more than one person is supplying relevance decisions, please 
follow exactly the procedure outlined in our previous instruction *eet 
for 'Relevant References Retrieved by the BA-Previews ;'ervioe« (BV-7). 
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Ezf«riMBtal laforafttioa Uait 
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ASCA SERVICE t INSTRUCTION SHEET 

GENERAL 

1. Th« next atftge of our •▼aluation will inTolyo a eoapariaoa of 
BA-ProTiowa with othor eoaputoriaod iaforaatioa aorrieoa. 

Ifor thoao OTalueoa whoao aubjoet iatoroata aro ia tho bioaodieal 
fiold, wo haTO eoaatruetod a MEDLARS profilo aad for thoao ia 
iho bioehoaieal field a Choaieal Abatraeta Coadoaaatoa profilo. 
Aa your profilo waa aot eoaaidorod auitablo for oithor of tboao, 
wo would liko to fivo you a eitatioa aoareh, uaiag ASCA 
(Autoaatie Subjtct Citation Alort), tho aacnotie tapo voraioa 
•f SeioBOo Citation Indox. 

2. Tko unaaaal naturo of tho ASCA aoareh ahould roault in tho 
rotrioTal of a aubataatial nuabor of artieloa which would not 
bo found by a aoareh of titloa, or of eouTontianal indoxoa, of 
iho typo aaod in tho BA-ProTiowa aoareh. 

3. Tho ASCA aorTieo allowa a aoareh to bo aado for eitod author a 
•r eitod roforoneoa. 

Citod author aoarehi For a firon author, thia will 
rotrioTO all eurroat artieloa whieh eito ia thoir 
bibliofraphiea an^ papor by thia particular aathor. 
(It will not retrioTO papora writtoa by thia author). 
Citod rof oroaeo aoareh i Thia will rotrioTO only 
eurront artieloa whieh eito thia particular roforoneo 
in thoir bibliographioa. 

INFORMATION REQUIRED 

4. fill you ploaae liat oa tho attaehod fora not loaa thaa 10, and 
not Boro thaa 16, citod authora aad/or eitod roforoaeoa. 

a) Citod authori a koy author who in tho paat haa 

publiahod work rolovant to your roaoareh work, and 

whoao papera are likely to bo eitod in eurront artieloa 
which are reloraat to you. 
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IN8THPCTI0N SHEET (CONT^P ) 



2. 



b) Cited reference : a specific reference by a key 
author* This should be given if any of your key 
authors has published in several research areas 
and only one or two of his papers are likely to be 
cited in papers relevant to your interests* 
Books (or specific chapters by individual authors) 
and other non-serial publications say be giveu^ 
but we would prefer that you restrict this type 
to no sore than 3 references* 

N*B* Your list of citations nay consist of authors 
onlyt individual references only^ or a Mixture of 
the two* 

5* The list of citations should cover all the research interests 
included in your BA-Previews profile* 

6« References published in 1970 or 1971 should not be included* 
(it is rare for references to be cited the year iuediately 
following publication)* 

7* There is no limit to the age of cited references* However^ as a 

general rule« references more than 10 years old are not often citedt 
with the exception of very important authors or papers* (This 
■ay not be true of some subject areas ^ e*g* classical taxonomy*) 

8* Will you please return the completed form in the enclosed 
addressed envelope^ as soon as possible* 
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Biperimental Information Unit 
7 Keble Road Oxford 0X1 3QL 

BA«PREVIEtfS PRQ.JEGy 
ASCA SERVICE ! CITATION 3EABGH 
INSTRUCTION SHgg f 

GENERAIf 

1. A few weeks ago you provided us with a list of up to 15 cited authors 
anVor references for a citation search. 

2. Ve had hoped to give you a computer printout of references using ASCA, 
the magnetic tape version of Science Citation Index (s.C.I.). However, this 
proved impossible. 

3. A manual search for the citations has therefore been made in the S.C.I, 
for January-March, 1971. 

4. On each of the enclosed cards is a reference which has cited one of the 
authors or references you gave us. 

INFORMATION REQUIRED 

5. Will you please mark each reference:- 

MAJ (Major Value) / MIN (Minor Value) / X (irrelevant) 

6. Each reference should be marked in the right-hand margin. 

7. We do £ot require you to indicate whether or not you already know of the 
'existence of the article. 

8. The cards should then be returned in the enclosed envelope, as soon as 
possible. 

9. If more than one person is supplying relevance decisions, please follow 
exactly the procedure outlined in our previous instruction sheet for "Relevant 
references retrieved by the BA-Previews Service". (bV-7) 
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Appendix IV. 5 

PROCEDUIffi FOR THE ANALYSIS OF COVERAGE SAMPLES 

A reference was classified as covered only if the enriched title and full 
bibliography had been located in either the abstracts of BA or the 
bibliographic listing in BRI . 

Assistance in locating references was provided by a pagination list which 
was generated by a computer search of the individual BA and BRI tapes for 
the period January 197I to February 1972 inclusive. This list was in. 
Journal coden, then volume number, then issue number, and then page number 
order. It provided a record of the page range of each journal which 
appeared in each issue of ^ or £21. Thus, a sample entry is: 

LYMPB LYMPHOLOGY 

5220 2 k 160 166 

5205 3 2 71 75 

This entry shows that issue 20 of BA (vol 52) contained entries from the 
Journal, Ij^phology, volume 2, issue no. 1+. The earliest page number to 
appear in the issue of BA was page I60 and the latest was 166. 

The pagination list provided an Indication of the probable issue of BA or 
BRI in which a particular reference in the coverage sample appeared. 
However, this list was not inerrant, nor did we assume it to be so. Also, 
we believe that a small but significant proportion of items from the 
coverage sample might appear in JA, or ^ either before or after the 11; 
month period covered by the pagination file. 

Items from Journals which had contributed no items to the pagination list 
were checked against the BIOSIS source list for I97O-72 to determine whether 
the particular Journal was covered at all by BA or mi. Items from 
Journals which were not covered were coded accordingly. 

For all other items which had not been identified as being covered by 
either BA or BRI, a search was made by means of the cumulated author 
indexes of BA and BRI for 1970 and 1971. In the case of all the alternative 
service samples and about one half of the BV-6 samples author indexes of 
the individual issues of JA and JBRI were searched for the period January 
1972 - September 1972 inclusive. Articles from the remaining BV-6 samples 
were checked against the 1972 author indexes for January and February only. 

In checking the author indexes as much care as possible was taken to avoid 
errors arising from alternative transliterations (e.r. from the Cyrillic 
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alphabet). 

7. Soma referenoes were also checked by the use of the Key-Word-In-Context 
(KWIC) indexes of BA and 

8. Inevitably human errors will have occiarred in using the author indexes and 
the pagination list. Purthexmore some itemp which were classified as not 
covered may in fact have been covered outside the period which was searched. 
Nevertheless, we are satisfied (of i^pendix IV. 6) that the great majority 
of items occurred during the period searched and, more particularly, in the 
1971 issue of Si or m. It will he noted that these issues were checked 
ty two laa^jfely independent methods - via the pagination list, and by means 
of the author index. Ve are therefore satisfied that the error rate in 
identifying covered references is unlikely to have a significant effect 

on our estimates of coverage ratios. 



o 
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Appendix IV 6 



TABLE: DISTR IBUTI ON OP APPEARANCE DATES IN BA PREVIEWS Or- 
REFERENCES PROM COVERAGE SAMPLES 



MONTH 


SOURCE OP 


SAMPLE 


MEDLARS 






OPT 


PAP 














Jan-March 


i 

1 


n 
u 


0 


1 


Aprll-Jioie 


1. 


u 


2 


8 


July* Sept 


J, 


n 
u 


0 


3 


Oct-Dec 




u 


2 


2 


1971 










J an 


11 

1 1 


u 


1 


17 


r6D 


5 


2 


29 


18 


March 


8 


1 




32 


April 


Q 


1 o 


75 


32 


May 


Q 

7 


oA 
^0 






June 


OR 


Q). 
74 


23 


68 


July 


*K 


7Q 

f7 


23 


75 


Aag 






8 


38 


Sept 




10 


ii 


18 


Oct 




0 


10 


23 


Nov 


7Q 


o 
c 


7 


17 


Dec 


i*3 


3 


6 


20 


1972 










Jan 


2i| 


3 


0 


8 


Peb 


9 


2 


2 


8 


Meirch 


0 


0 


1 


3 


April 


2 


0 


1 


7 


May 


2 


0 


0 


ii 


June 


2 


0 


1 


0 


July 


0 


0 


0 


2 


Au^ 


0 


0 


0 


1 


Sept 


0 


0 


0 


2 


Total 


k93 


292 


280 


1|82 



Hie above table shows the chronological distribution of references in 
each of the four coverage samples which were covered by either Biological 
Abstracts or BioResearch Indexr^'For each samplei the number of references 
appearing in each month of BA or BRI is given* 

Date for the ASCA^ CAC and MEDLARS samples are based on all references 
in the coverage samples which were covered by BA or BRIe The data for the 
BV-6 sample (•references found by the userO for a subset of 1^93 of the 950 
references covered by the data base* 
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The xeiezenoes in the alternative service san^lea corresponded to the 
following Issues of these services t 

MEDLAIiSx April- June, 1971 

CACt Vol 7U; Issues 5f Tt St 11 (published in 
Februaxy/Harob, 1971) 

sell Januaxy-March, 1971 

saoples consisted of articles fo\md by the user in June, July and August* 
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Currency s pread in T^A/SIJ for individual issues of primary .ioumals 



A random sample of 2';o individual issues of primary journals was selected. 
For each issue, we identified the number of months of BA or BRI over which 
articles from that issue were distributed, i.e. a 'spread' of 5 months 
indicates that there was a gap of 5 months between the earliest and latest 
dates on which articles from a given journal issue appeared in BA/BRI, 

The resTJlts are shown in the following table. For example the table 
shows that an individual journal issue was spread over 3 months of BA/BHI 
in 38 of the 250 issues in the sample* 



TABLE} CUBHENCY SPREAD IN BA/BRI 



Currency Spread 
(Months) 


No. of 
Occurrences 


% of 
Occurrences 


1 


I46 


18.1+ 


2 


$h 


21.6 


3 


38 


15.2 


. h 


37 


1U.8 


5 


6 


2.U 


6 


2 


0.8 


7 


12 


U.8 


8 


12 


U.8 


9 


17 


6.8 


10 


16 


6.U 


11 


9 


3.6 


12 


1 


0.1+ 




250 


100.0 
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tablej serial publications not covered by biological 

ABSTRACTS OR BIORESEARCH INHEX 

The following table gives the 86 journals which contributed one or more 
relevant items to the coverage sampleB but which were not covered by BIOSIS 
in 1971. 

A journal codeu is given for those titles which could be located in the 
ASOU listing. Journals for which no coden could be located are indicated as 
follows: . 

* the jouinal title was identified in one of the standard serials 
listings (Ifeion List of Serials, Current Serials Received by the NLL, 
etc) and there is no doubt m to its identity. 

? (11 titles) the title could not be found in any serials list. There is 
a possibility that either the title is incorrect or that it does not refer 
to a serial publication. 

Journals contributed relevant items to ^e user only with the exceptions 
indicated by the figure in parenthesis after the ooden, e.g. AVFZA(2)t this 
jwumal contributed relevant items to coverage samples from two users. 
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Journ€Ll Title 


Coden 


Acta Parasltologlca Llthuanlca 


APLUA 


Aliment et la Vie 


ALLVA 


American Corrective Therapy Joumal 


* 


Archlv fuer Pf lanzenschutz 


AVPZA (2) 


Arkady Review 


AKBYA 


Attl della Socleta Itallana Bulatrla 


* 


Blokhlm Immunlteta Pokoya Rast 




Brauwelt 


BRUWA 


Bxxlletin of the American Ceramic Society 


BACSA 


Bull Fac Domestic Scl^ Otsuma University 


« 


Bull Jap Ent Acad 




Bulletin of the Japan Society of Mechanical Engineers 


BJSEA 


Bulletin de Madagascar 


BUMTA 


Bulletin of Prosthetics Research 


BPBBB 


Cell\iLar Immunology 


* 


Ceskoslovenska Neurologle 


CSKNA 


Ceskoslovenska Zdravotnlctvl 


CEZBA 


Chemical Technology^ A Series of Monographs 


CHINA (3) 


Chemist and J)rugglst 


CHBRA 


Current Topics in Pathology 




Deutsche ZeitschTif t •Put* VerdauuniTfl und Stoffwechselkrankhelten 


BZVSA 


Dlagn Potrebnostl Rust udobr 


r 


Sojciaoy Kazaicnsjcoi Aicaaemii rei/bKOicnozyaistvennyJcn j)«auK 


TMT AC A 


DoKiaoy Bulgarsxa AKaaemii na Naulcite 


TYD AU A 


Essays Toxicol 


o 

r 
« 


Flsherj' Bulletin of the National Oceanic and Atmospheric 


•N- 


Administration 




Plzlol Aktiv Komponenty PI tan Zhlvotn 


? 


Flzlologlya 1 Blokhlmlya Rastenll Akademll naul. Belorusskol SSH 


FBIBA 


PI el schwl rt shaf t 


PLEIA 


Pood World 


POWBA 
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Jouxnal Title 


Coden 


Gen En« (India) 




Genetika a Sleohtenl 


GESLB 


Geoohemistxy Intexnatlonal 


GCTKA 


Geoohimloa et Cosmoohimioa Acta 




Geoteohnloa 




I.S.I. Bulletin (Indian Standards Institution) 




Industxia Alimentara 


TWAT.A 
X£i H hit 


Industries ilimentalres et A^iooles 


lALAA 


Inland Aoad Soi (Indiana) 


« 


Int Journal of Parasit 


• 


Izvestiya na Mikrobiologioheskiya Institut. . . 




Japan igrioultural Besearoh Quarterly 


# 


Japanese Society of Internal Medicine - Journal 


AJIMA 


Journal of the Agricultural Association of China 


JIGIA 


Journal of the Institution of Chemistry 




Journal of the SocieV of Diairy Technology 


JSIVPA 


Journal of Sports Medicine and Physical Fitness 


JMPPA 


Kagaku No ^oiki Zokan 


KNR2A 


Kharohova Promislovist, Naukovo-Tekhnichnii Zbimik 


KPTZA 


Kvasny Prunysl 


KVPRA 


Lambillionea 


LMBLA 


Minerva Ginecologica 


MTCTA 


Minerva Nipiologica 


MTHIA 

A 1 AA^ AA 


Nederlandsch Tijdschrift voor Verloskunde en Gynaecol ogie 


NTVGA (2) 


Norv Gumoral Mekh Hegul Funkts 


• 


Nippon Sojo-Hiryogaku Zasshi 


NIDHA 

Ai Ml Art i Jl 


Nippon Sakumotsu Gakkai Kiji 


NISAA 


Pesticide Science 


# 


Poljoprivredna Znanstvena Smotra 


PJZSa 


Problemy Parazitoiogii 


PPUZA 


Progress in Phytochemistry 


PRPCB 


Regul Rosta Rast Khim Sredatvami 


• 


Revista Chilena de Obstetricia y Ginecologia 


RCOBA 


Ricerca Scientifica, Revista 


RSHVA.(2) 


Royal Society of Health Jo\imal 


RSHEA (3) 
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Journal Title 


Cod en 


Scaoidinavian Journal of Dental Research 




bedimentary Geology 


SEGEB 


Seibutflu-Butfluri 


SEBUA 


Studia Biophysica 


STBIB 


Studii si Cercetari de Medicina Interna 


SCMDA 


Suplemento Cientifico^ Consejo General de Colegios Veterinarios 




de Espaiia 




Teor Vop Obrab Pochv 




iomoo 




Triangle 


TRGLA 


Tropical DieeaeeB Bulletin 


TDBUA 


Vestnik MoBkovekogo Univereiteta Seriva II Khimiva 


VWTT A 

VWJvUA 


Vestnik Stxxdencheskogo Nauchnogo Obshchestva* Kazanakii 


vua.ua 


GoBudarstvennyi Univereitet Im V.I. Ul/Yanova-Leriina 


Vestsi Akademii Navuk Belaruekai SSR. Seriva Bivalao-irhrivVVi 


VAi}i5A \ 


Navuk 


Veteriner Pakulteei Dergisi 


VTPDA 


VoproBy Piziologii i Biokhimii Kul/Tumykh Eaetenii Akademiya 


VPBRA 


Nauk MoldavBkoi SSR, Inetitut Piziologii i Biokhimii Rastenii 


Valdhygiene 


WLDHA 


ZeitBchrift fuer Geburtshilfe und Gynaekologie 


ZGGNA (2) 


Zeeyty Naukowe Uniwersytetu V Toruniu, Nauki 


ZTMBA 


Matematyczno-Przyrodnicze-Biologia 


Zhum/il Analiticheekoi Khimii 


ZAKHA 


Zhumal Organicheekoi Khimii 


ZORKA 


Zivotnovadni Nauki 


* 
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Ap-pendlx V 

CONMTS 



1. FINAL QUESTIOMAIBE 



2. ADDITIONAL ADVANTAGES AND DISADVANTAGES OP THE BA PREVIEWS SERVICI 
WHICH WERE MENTIONED BY TISEBS IN BEPLY TO QUESoOTl 

3. TABLEt COEEELATIONS BETWEEN USERS' REPLIES TO QUESTIONS 12 AND 13 
AND REPLIES TO QUESTIONS 1-11 

l» CARD OUTPUT: [QUESTIONNAIRE] 
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Experimental Information Unit 
7 Keble Road. Oxford 



BA-HIEVIEWS PROJECT 



FINAL qUESTIONNAIRE ^py 
Profile No: Name: 



1 . We should like to find out what you consider to be the main advantages 
and disadvantages of the BA-Previews service. For this purpose we 
have listed below some of the possibilities. Will you please tick 
only those which you consider to be really IMPORTANT to you. Do not 
tick any which are only of minor significance. You may tick none, 
one or several in each list. Others may be added in the spaces 
provided. 

ADVANTAGES 

a) The profile retrieves references of major value which I 
might not otherwise locate. 

b) The profile retrieves background references which I 
mi^t not otherwise locate. 

c) The service covers journals which I \^ould not normally see. 

d) The service retrieves references from non-journal 
publications, e.g. books, symposia, conference proceedings, etc. 

e) The service is time-saving, i.e. it reduces the time spent 
searching the literature by normal methods. 

f) The service is convenient, i.e. it obviates the need for 
copying out references from abstracting and indexing journals 
for incorporation in my personal index. 

g) The provision of a regular service gives me confidence; my 
literature searching is no longer dependent upon how and 
when I can do it myself. 



Others (please specify):- 



h) 



i) 

□ 



/cont. . . 
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DISADVANTAGES 

j) The profile retrieves too many irrelevant references 

k) The profile misses too maiy relevant references 

l) There is too much delay between publication of an article 
and its appearance in BA-PreviewB 

m) The journal coverage is inadequate 

n) Many relevant references are of little or no practical value: 
(i) They are from journals which are not easily accessible 

(ii) They are from publications which I consider to be of 
low quality 

(iii) They are from foreign language journals which I 
cannot read or afford to have translated 

o) The computer printout is unattractive 

p) The computer printout is difficult to file 

q) The coding of the profile is difficult to understand so 
I am not sure how the profile logic is intended to work, 
i.e. which single search term or combination of search 
terms will retrieve a reference 

r) There is not enough information on the printout: 

(i) no abstracts 

(ii) no author addresses 

(iii) no indication of the language of the original 
paper 

Others (please specify)}- 



BA-'^vievB Project t Queetionnalre 

Note t Questions 2, 3 and 4 (below) refer to disadvantages listed in 

Question 1 . However, we would like a direct answer, irrespectiT* 

of whether you have ticked them as 'Important Disadvantages* . 

2. Do you consider that your profile retrieves Tes 
too many irrelevant references? 

No 

DonH know 

3* Do you consider that your profile misses too Tes 
masy relevant references? 

No 

Don't know 



4. Do you consider the journal coverage of Adequate 
BA-Previews for your research interests is: 

Inadequate 
Don't know 



5. Do you find it difficult to decide, from the Frequently 
title alone, whether an article is relevant 
to your research interests? Occasionally 

No 

Don't know 



6. The use of a liaison scientist to provide 

assistance in the construction and amendment of 
profiles Is expensive. Furthermore, some research 
workers may not wish to delegate this task to 
someone else but would prefer to construct their own 
profile. Would you prefer 

a) To construct your own profile, after receiving the necessary 
instruction on the technique of profile construction, and 
have it checked, or 

b) To have the profile constructed by a liaison scientist, as 
at present. 
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BA-PreviewB Pro.iectt Queatlonnaire 



4 



7. 



8. 



9. 



10. 



Since you started receivttg; BA-Previews computer printouts, has 
there been an Increase or decrease in your use of the printed 
Biological Abstracts for Current Awareness anVor Retrospective 
searching? 





Current Awareness 


Retrosneotive 


Increase 






Decrease 






No change 






Don^t know 






Never use it 







After selecting the relevant references on your 
computer printout, do you subsequently look up 
the original articles, provided you have not 
already seen them? 



Have you experienced difficulty in obtaining 
the original articles? 



Always 
Frequently 
Occasionally 
Never 

Always 
Frequently 
Occasionally 
Never 



Some participants have asked if there will be 
a permanent BA-Previews Cuzrent Awareness service. 
If there were such a service do you consider that you 
would be able to reduce the time spent:- 

a) Scanning current Journals 



h) Scanning secondary journals, i.e. 
abstracting and indexing journals and 
titles lisbs 



Considerably 
Slightly 
No 

Don't know 
Considerably 
Slightly 
No 

Don*t know 



□ 
□ 
□ 

□ 

□ 
□ 
□ 
□ 



□ 

□ 
□ 

□ 
□ 
□ 
□ 
□ 
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BA-Previews Pro.lect: Questionnaire 



11* Iff in addition to the Current Awareness service Greater value 



it became possible to operate computer searches 
of Biological Abstracts for the last 1-5 years, 
would this Retrospective search service be of 
greater or lesser value to you than the 
existing Current Awareness service? 



The same value 
Lesser value 
Don^t know 



12. If there were a permanent BA-Previews Current 
Awareness service would you wish to become a 
subscriber, assuming that your department/ 
institution would pay the subscription? 



Yes 

• No 
DonH know 



^3• If a permanent BA-Previews Current Awareness 

service were established in the UK, on a full cost- 
recovery basis, the minimum cost of a profile is 
likely to be £50-JC75 per annum (though improvements 
in search programs might reduce this substantially 
during the next few years). Do you consider your 
present BA-Previews profile to be worth £50-£75 per 
annum? 



Yes 
No 

Don^t know 



14. How many people (including yourself) see your 
printout regularly? 



FURTHER COMMENTS:- - 
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Additional advantages and disadygAt akea of the BA Prevlewa Bervloe whlo^ 
were men tioned by uflare In reply to Queation 1 . 

Ir. the final queationnaire uaera were provided with liata of both 
advantagea and diaadvantagea and were aaked to tick thoae which they 
oonaidered to be applicable. They were alao given the opportunity to 
nominate advantagea and diaadvantagea other than thoae liated. Only 
three advantagea and three diaadvantagea were apecified in the »o there* 
category and theae are liated below. 

Advantagea 

1. "In conpariaon with UKCIS [CA-condenaatea aervice], the uae of 
enrichment terma enablea me to identify more efficiently articlea 
of major intereat." 

2. "ThB aervice bringa to my notice rapidly developing techniquea, 
materiala, etc. In fielda other than thoae in which I work and which 
may be uaeful in ny own field. " 

3. "Olie aervice acta aa a recurring atimulua to look at current 
publicationa more frequently than I would otherwiae." 

Diaadvantagea 

1. "Inaccuraoiea in the printout, auch aa miaquotationa of authora namea 
• and confuaion of taxonomic tezma, eapecially aimilar generic namea, 

are frequent and reduce confidence in the ayatem." 

2. "There are poor tranalationa of foreign-language titlea, eapecially 
of technical texna." 

3. "It ia not alwaya poaaible to tell in which country the work waa done." 
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TABLE: COKBELATIONS BETWEEN USERS' EEPLIES TO QUESTIONS 
12-13 AND BEPLIES TO QUESTIONS 1-11 



QUESTION 


QUESTION 12 


QUESTION 13 


% 


% 


(X 


% 




a 


Q1 


Advantages 
















1(a) 


YES 


60 


21 


0.0001 


62 


39 


0.0001 




Kb) 


YES 


76 


<R 
50 




7). 


00 






1(c) 


YES 


U 1 








7A 


C0.97; 




1(d) 


YES 






).n^ 


-57 




kO.03; 




1(e) 


YES 


Ji7 


pn 


o ooni 










1(f) 


YES 




1 






in 


U« UU1 




1(«) 


YES 




1 0 


U«UUU1 


1.7 




0*000 1 


01 


Disadvantflutes 
















i(j) 


YES 


39 


52 


0.05 


32 


50 


O.CX)3 




i(k) 


YES 


9 


12 


(0.60) 


5 


13 


(0.07) 




1(1) 


YES 


kS 


52 


(0.30) 


39 


53 


0.03 

1 




Km) 


YES 


8 


12 


(0.50) 


9 


10 


(1.0) 




l(n- 


i) YES 


16 


20 


(0.55) 


12 


21 


0.06 




l(n- 


ii) YES 


9 


7 


(0.69) 


8 


9 


(0.9U 




l(n- 


iii) YES 


22 


21 


(0.98) 


15 


26 


0.05 




Ko) 


YES 


29 


26 


(0.67) 


32 


25 


(0.27) 




1(P) 


YES 


h2 


3h 


(0.26) 


k3 


37 


(0.14+) 




l(q) 


YES 


11 


10 


(0.90) 


11 


11 


(0.88) 




l(r. 


i) YES 


29 


21 


(0.18) 


26 


28 


(0.83) 




I(r- 


ii) YES 


30 


21 


(0.16) 


27 


28 


(0.96) 




l(r- 


iii) YES 


ho 


30 


(0.17) 


35 


38 


(0.79) 
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BEST COPY AVAllABLE 



TABLE (CONT'D) 







QU 


ESTION 12 


QUESTION 13 




QUESTION 


YES 

0/ 


NO 

% 


a 


YES 

% 


NO 

% 


a 


02 


YES 


hs 




0.01 


39 


60 


0.001 


Q3 


YES 


13 


11; 


(0.96) 


8 


17 


0.05 


Qh 


ADEQUATE 


Ih 


69 


(O.iil) 


76 


70 


(0.33) 


05 - 


FREQUENTLY 
OCCASIONALLY 


13 
80 


20 
83 


(0.72) 


1 0 

79 


1 7 

82 


1.0.21;; 
(0.60) 


Q8 


ALWAYS 

ALWAYS/PEEQUENTLY 


16 
71 


20 
63 


(0.55) 

(0.21) 


17 
73 


17 

66 


(0.97) 
(0.29) 


Q10 (a) 


or (b) CONSIDERABLY 


70 


33 


0.0001 


73 


i;9 


0.0001 


Q11 


GREATLY 








35 


i;7 


(0.06) 



Notes to Table* 



(i) the question numbers listed in the left-hand column are these used 
in the final questionnaire (See Appendix V.l) 

(ii) the percentages indicate the proportion of those users givin,? a 
particular reply to question 12 or 1 3 who also gave the reply 

column. E.g. the first entiy in the column 
headed 'QUESTION 12 - YES' indicates that 60% of users replying 
'Yes: to question 12 had also ticked advantage 1(a) 

(iii) the significance a was computed from values of x^. A correlation 
was considered to be 'not significant' if a - 0.05 and the value of 
0 IS shown in parentheses (two-tailed test) 
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Experimental Information Unit, 
7 Keble Road. Oxford 0X1 3QL 



BA-PREVIEWS ! CARD ODTPUT 

Puring the course of the BA-Freviews experiment, some considerable 
interest has been shown by participants in the possibility of the computer 
printout being available on cards as opposed to paper. 

In order to demonstrate the alternative format, this issue of BA -Previews 
has been printed on cards. If the BA-Previews service continues for a 
further experimental period, it is likely that there will be a choice between 
card or paper output. 

We should like to have your opinion of this card output and would be 
grateful if you would kindly tick the appropriate box below and return this 
form to the above address. 



'1. I prefer the card output 
2. I prefer the paper output 




3. I have no preference for cards or 
paper 
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Apperdix VI 

CONTENTS 

INFOBMATION CMTIiES: FINAL QUESTIONNAIBE 
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Experimental Information Unit 
7 Keble Road 



BA-PRETIEWS PROJECT 
mPORMATION CENTRES 
FINAL QUESIIQMNAIRE 

Profile No: Name: 



1. We should like to find out what you consider to be the main advantages 
and disadvantages of the BA-Previews service. For this purpose we 
have listed below some of the possibilities. Will you please tick 
only those which you consider to be really IMPORTANT to you. Do not 
tick any >diich arw only of minor significance. You may tick none. 
one or several in each list. Others may be added in the spaoes 
provided. 

APyANTAGES 

a) The profile retrieves references >rtiich we might not otherwise 
locate. 

b) The profile retrieves articles from ;Joumals vrtiich wo do 
not cover in our eidsting literature searching. 

* 

c) The service retrieves references from non-;Jouxnal publications, 
e.g. books, symposia, conference proceedings, etc. 

d) The service is time-savinT* i.e. it obviates the need for 
manual searching of the hard copy Biological Abstracts. 

e) The service is convenient, i.e. it obviates the need for 
copying out references from secondary ;Joumals, as the print- 
out can be used for processing our bulletin/ index. 



Others (please specify):- 



f) 



e) 



□ 
□ 

□ 
□ 



□ 
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g) The profile retrieves too many irrelevant references. 

h) The profile aliases too many relevant references. 



i) There is too much delay between publications of an article 
and its appesirance in BA-Previews. 



2. 



o) 



P) 



j) The journal coverage . is inadequate. 



k) The computer printout is unattractive. 

l) The computer printout is difficult to file. 

m) The coding of the profile is difficult to understand. 

n) There is not enough information on the printout: 
(i) no abstracts 

(ii) no author addresses 

(iii) no indication of the language of the original 
paper. 



Others (please specify):- 



□ 

□ 
□ 

□ 
□ 

□ 

□ 
□ 

□ 



□ 
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3. 



BA^Prevlews Pro.iectt Questionnaire 



Notet Questions 2, 3 and h (below) 3?efer to disadvantages listed in 

Question 1. However, we would like a direct answer, IrreBpective 
of whether you have ticked them as 'Important Disadvantages'. 



2. 



3. 



5. 



6. 



ERIC 



Do you consider that your profile retrieves 
too many irrelevant references? 



Do you consider that your profile misses too 
many relevant references? 



Do you consider the Journal covera^ of BA« 
Previews for your Information Centre's 
interest is: 



Yes 

No 

Don't know 
Yes 

No 

Don't know 

Adequate 

Inadequate 

Don't know 

Frequently 

Occasionally 

No 

Don't know 



The use of a liaison scientist to provide 
assistance in the construction and amendment of 
profiles is expensive* Furthermore, you may not 
wish to delegate this task to someone else but 
would prefer to construct your own profile. Would 
you prefers- 

a) To construct your oim profile ^ after receiving the necessary 
instruction on the technique of profile construction, and 
have it checked, or 

b) To have the profile constructed by a liaison scientist, as 
at present* 



Do you find it difficult to decide, from the 
title alone, whether an article is relevant 
to the interests of your Information Centre? 



□ 

□ 

□ 
□ 

□ 
□ 

□ 
□ 

□ 
□ 

□ 
□ 
□ 



□ 
□ 



/oont«« 
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ftuestionnaire 



8. 



9. 



Since you started receiving BA-Previews computer printouts, has there 
been an increase or decrease in your use of the printed Biological 
Abstracts for Current Awareness and/or Retrospective searching' 





Current Awareneaa 


RetrosDeot^ve 


Increase 






Decrease 






No change 






Don^t know 






Never use it 







Some participants have asked if there will be a 
permanent BA-Previews Current Awareness service. 
If there were such a service do you consider that 
you would be able to reduce the time spent by your 
Infozmation Centre: 

a) Scanning current journals 



b) Scanning secondeuty journals 



Considerably 
Slifi^itly 
No 

Don't know 
Considerably 
Slightly 
No 

Don't know 



If, in addition to the Current At/areness service Great . r value 

it became possible to operate computer seeujches 

of Biological Abstracts for the last 1-5 years, The same value 

would this Retrospective search service be of 

greater or lesser value to you than the existing Lesser value 

Current Awareness service? 

Don^t know 



□ 
□ 
□ 
□ 
□ 
□ 
□ 
□ 

□ 
□ 
□ 
□ 
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BEST COPY AVAIUBLE 



10. If thoT' v;rfre 'i p ^rmanont BA- Previews Current 

Awaronf.r:: cnrvic^ would you wish to become a yes 
subt:c:ri b^r» 'k-.'m- : >;,- that adequate funds were 
availablr*: 

No 



DonH know 

11. If a pcrrnanont BA-Previews Current Awareness 

service wore octablished in the UK, on a full Yes 
cost- recovery basin ^ the minimum cost of a 
profile ii: likely be £50-£75 per annum 
(though improvomont,;; in search programs might No 
reduce this substantially during the next few 
years). Do you consider your present BA- 
Previews profile to bn worth £50-£75 per annum? Don»t know 



12. We should like to know what usei if any, you have been 

making of your BA-Previews printout during the experiment. 
Some possible uses are listed below. Will you please tick 
those which apply to your Centre and also supply the 
missing details, as indicated. Others may be added in the 
spaces provided. 

a) *All/Some of the relevant references are selected for 
incorporation in a published ^Abstract Jouinal/lnformation 
Bulletin, which is distributed to subscribers. 

b) *All/oori'o of thn relevant references an- looted for 
incorporation in an Information Bulletin, v/hich is 
distributed I'roo tc- workfin:. 

c) ^All/Uome of the relevant references ar. .-.rlocted for 
incorporation in thi Tni'-rmation Centre*: -^wn card index. 

* Delete whiehevf?r does not apply 

d) None of the ro! erences on the printout h;.vp been used by 
the Info rma Moil Centre. 

Others (ri- .•f^'- i ^v) : - 

e) 



O cont/ 
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